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Dr. H. Lee Martin, President 
TeleRobotics International, Inc. 
8410 Oak Ridge Highway 
Knoxville, Teunessee 37931 

Dear Dr. Martin: 

We have looked at your proposed electro-optical image orientation system 
that you are presently developing under NASA contract and are very 
interested in its potential. Your exploratory* efforts in the develop- 
ment and implementation of the algorithm that transforms a fisheye image 
into multiple standard views appear to verify that your concepts have a 
practical realization. 

We are interested in developing and funding a commercialization plan for 
this product contingent upon the technology being reduced to practice 
and perform the image transformations at a rate compatible with live 
imaging (30 frames per second) . We would anticipate entering nego- 
tiations with you concerning your specific funding needs for cotmercial- 
ization and product inventories one year after the beginning of your 
Phase II development effort. At that point in time you should have a 
focused estimate of the fabrication costs, inventory costs, and the 
initial target marketing plan. 

The potential of the electro-optical viewing system for surveillance and 
navigation appears large. I would recaimsnd that you augment your 
copyright protection with patents where possible, but the Application 
Specific Integrated Circuit that you mentioned should also be a strong 
barrier to entry. I look forward to further progress on your develop- 
ments. 




Sincerely, 

U. S. ENTERPRISES, INC. 
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Optimizing the Camera and Positioning System for Telerobotlc Work site Viewing. 

TECHNICAL ABSTRACT (LIMIT 200 WORDS) 

As NASA implements manipulation equipment that removes the astronaut from the direct 
worksite, the ability to properly view the workspace is the single most critical issue 
to successful teleoperated remote manipulation. With NASA's growth plan from tele- 
operation to autonomous operations, the viewing system must be conceived to serve both 
needs. This effort will develop a concept and prototype viewing system that will 
maximize the camera positioning/orientation capabilities while simplifying the 
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A "smart" camera interface will be concepted that allows the camera signal transmission 
to be controlled in the following modes: frame rate (variable depending on importance 
of view), resolution (variable depending on type of view), and information compression 
(normal scan, differences only, levels only). A second pan/tilt method based on • 
electro-optics will also be explored. 
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The mechanical concept has potential in earth-based teleoperation activities in 
nuclear and defense applications. The commercial potential of the electro-optical 
pan and tilt is significant. Kany stores and banks have a camera on a pan and tilt 
logging scans of- the facility for security purposes. The device described has the 
potential to be less expensive, more versatile, and more reliable than the conventional 
^nirmpn-h now in place opening a_ large^QJti&M al market for the device. 
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Technic al Content 
n identifica tion Significance of thft Innovation 

Over the next decade. NASA seeks to develop the equipment necessary to allow manipulators to 
be remotely deployed for very dexterous tasks. These manipulators will initially be teleoperated 
and then evolve into autonomous devices able to accomplish construction of space structures and 
satellite servicing. During this evolution of technology, the importance of visual feedback will 
become evident, first for good human interaction via teleoperation, and then for machine sensory 
fusion to guide autonomous operations. Thfi visual information channel iff the single most important 
ff .Hh a Hc rhannei a" a n a hl« to the « manipulation device. Yet, the camera positioning 

system is usually one of the last devices to be added to an already cluttered interface platform. With 
NASA strongly considering the use of 7 degree of freedom manipulation systems, the camera 
positioning problem becomes even more challenging as we have discovered in recent tests with a 
kinematically redundant manipulator mockup. Now the camera positioning system must be able to 
provide viewing locations that are not subtended by the equipment in the worksite between the 
camera and the manipulators. Two commonly considered alternatives are jointed with actively 
driven camera positioning appendages and end-effector mounted camera systems. These 
alternatives suffer from complexity, poor resource utilization (i.e., reducing the available arms 
for work, or providing a motion picture to the operator), and large structural size that makes it 
difficult to get the camera positioned in the right location. The innovation to be developed has 
several elements to address the problems of remote viewing, each with its own purpose. The basic 
goals of the innovation are: 

1 Create a positioning device that can place the camera anywhere in the working 
volume of the manipulator and hold it there while the arms are free to work. 

2 Create a pan and tilt device that is mechanically self-contained and therefore 
requires no consideration for obstacle avoidance once the unit is in position. 

3. Create a camera control and signal interface that is flexible to minimize the 
transmission burden on the communications system. 

The goals are commendable, and the simplicity of the proposed solution may be surprising. Over 
years of developing and maintaining these types of systems, we have found that the Keep It Simple 
approach is often vindicated. The positioning support system is a passive cylindrical shape 
retention device approximately 4 ft long. In simple terms, it is a gooseneck with a camera on one 
end and a mounting base on the other. The manipulator is used to position the camera end and 
orientation is accomplished by a miniature internal pan/tilt device. As a result of its flexib.l.ty 
and small size, the device can be positioned anywhere in the manipulator workspace, yet rema.n 
stationary when the manipulator is in use. The advent of smaller cameras makes it feasible to 
actually mount the entire camera on the end of the support (even in 1g environments) ,f a small 
pan and tilt device can be developed. 
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Two potential solutions to the pan and tilt challenge will be explored. The mechanical solution 
is patterned after a much larger French solution developed 3 years ago. This approach places the 
camera inside a can and uses either a prism or mirror to accomplish miniaturized pan and t.lt 
capabilities. Figure 1 gives the basic idea behind the positioning and orientational concept. Due to 
the shear size of the physical hardware that was utilized by the French, their solution ,s not 
satisfactory for this application, but a smaller system is deemed feasible with available hardware 
components. One critical issue to the size is the angle of motion the pan/tilt must provide. Careful 
determination of this range will be performed prior to the design of the device. 

Shape Retaining "Goose-neck" 
Camera Positioner. 




Self-contained 
Mechanical 
Pan And Tilt 




Mirror 
\ 

Tilt Drive 
Lens 
I 

Pan Drive 
Camera 



Figure 1 - Positioning and orientational concept. 

A second method o. accomplishing pan, tilt, and in addition zoom, is with the use of 
elec.ro-optics. The physical concept is relatively s.raightforward. A high reso lu on 
(,024X1024 elements) digital camera is attached to a "Ash-eye" lens allowing a h,gh resolution 
scan o. a hemisphere. A drgita, scan controller selects the pixel information necessary ro. , e 
hemisphere to construe, the desired display view a. NTSC standard resolu ,on (33 x240 
elements,. One disadvantage tha. the fish-eye lens has in analog implementations . *■ ~n 
of the real world view In a manner similar to the relation between polar and rectangular 
coordinates. The digital processor wi» also be used to implement algorithms deveioped to remove 
mis distortion. Figure 2 shows the basic hardware elements of the concept. To accompli pan 
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tilt, the displayed area is shifted throughout the viewing field as shown in Figure 3. Zoom can be 
accomplished by proper selection of sparse pixels to zoom in and out as shown in Figure 4. A 
detailed conceptual design of this type of camera system will be performed during Phase I of the 
activity. 
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Direction 
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Control 



Figure 2- Hardware conceptual diagram for 
electro-optical pan, tilt, and zoom system. 

The final objective of this systems approach to viewing is to minimize the burden which 
viewing systems place on the remainder of the controls, power, and communications subsystems. A 
smart camera controller that accepts high level serial commands and services all camera functions 
(pan, tilt, zoom, focus, iris, light intensity, camera power, and light power) would allow easy 
addition of multiple cameras to the system as needed rather than having a firmly fixed set that later 
turns out to be lacking for some activities. This also unburdens the higher level control systems 
from controlling low level functions such as drive voltage. The other advantage of a smart camera 
interface is the ability to perform preprocessing on the camera signal information. At levels much 
lower than feature extraction, there are functions/modes that the camera interface can accomplish 
that will unburden the ultimate performance of the communications system and other viewing 
processing systems. Modal operation such as frame rate control (variable depending on importance 
of view), resolution selection (variable depending on type of view), and basic information 
compression techniques (normal scan, differences only, levels only) can greatly reduce the 
communications burden. These and other modes will be considered in the first phase leading to a 
prototype development in the second phase. 
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Figure 3 - Movement of standard resolution display throughout 
high resolution image produces electro-optical pan and tilt. 



1024 



512 



256 




1024 



0 256 512 

Figure 4 - Shading represents the "pixel packing" required to get 
automatic zoom from the Electro-optical lens system. 
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h Phase I Technical Objectives 

Several technical accomplishments are planned for Phase I: 



1 . Hardware realization and delivery of a passive camera positioning system with 
excellent worksite coverage potential. 

2. Hardware realization and delivery of an active, self-contained camera 
orientation device based on mechanical movement of the camera optics. 

3. Conceptual design development of a hemispherical viewing method based on 
electro-optics with no mechanical mechanisms to accomplish pan. tilt, and zoom. 
Successful implementation would be the focus of follow-on efforts. 

4. Conceptual design development of a smart camera interface that improves the 
ability to control, communicate, and add cameras to a working system. Successful 
implementation would be the focus of follow-on efforts. 

Past experience with numerous teleoperated manipulator systems has shown us that viewing 
is of the utmost importance and that it is also the most often overlooked by systems developers who 
are concentrating on the manipulator arms and not the entire manipulation system. An early 
mockup attempt of the positioning device proposed using 2 inch "diameter flexible conduit and optic 
fibers (H. L. Martin, 1984) demonstrated that the manipulator could readily position the camera 
in preparation for work tasks. Difficulties with the early attempt included: 1) the mechanical 
conduit was not rigid enough to maintain the set position due to its own weight and the weight of the 
optic fiber bundle (1/2 inch diameter and six feet long). 2) the manipulator was not adept at 
orienting the device once it was located (but this had to be done because the optic fiber bundle was 
limited to pointing in the direction of the support conduit, 3) the quality of the view through an 
optic fiber bundle was not comparable to direct viewing (lower resolution, lower light level, 
missing pixels due to breakage). The methods proposed herein directly address each of these 
problems using more recent technology. By mounting a miniature camera (which have both 
improved in performance and reduced in size and weight since the initial effort) at the end of a 
non-hollow flexible conduit, the system weight is reduced and the conduit strength is increased. 
The addition of active pan/tilt capabilities allows the orientation problem to be eliminated as long 
as the pan/tilt method is self-contained. The viewing quality will now be comparable to the quality 
provided by any other camera as no additional optical transmission devices are used. 

The development of a small pan/tilt method is fundamental to the success of this innovation. 
The mechanical method is straightforward as described earlier and no technical difficulties are 
foreseen. The electro-optical approach which offers the ultimate solution is more difficult. In 
previous efforts, (P. E. Satterlee. 1985) an analog viewing system was developed which 
demonstrates the ability to perform electronic tilt through a 90 degree conic angle and provide 
straight ahead or near area viewing for a mobile vehicle. The method eliminates the need for any 
moving parts in the orientation of the camera, but is limited to the 90 degree range of motion. The 
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next stage of the development as proposed for follow on work is to implement the concept in a digital 
system to increase the range of the viewing area, allow pan. tilt, and zooming capabilities, and 
provide an undistorted view to the operator. The most critical issue to the success of this effort is 
the speed with which the controlling microprocessor can access the camera information, reformat 
this information, and drive a monitor display unit. The controlling processor will be required to 
address 93.120 pixels of information in less than 1/30 of a second and produce a new pixel map at 
least every 100 milliseconds. The efforts in the first phase are to locate parts and conceptually 
design such a device. Once accomplished, the goals for the fourth objective become straightforward, 
to add intelligence to the command channel so that multiple operational modes can be performed. 

r. Phase I Work Plan 

The contractor shall perform research and development related to positioning, orienting, and 
controlling the camera system. Specific efforts will be focused on fabrication of a mechanical 
positioner and a self-contained camera orientation device with related development of designs for an 
electro-optical pan. tilt, and zoom system with no moving parts. A final effort will involve the 
conceptual design of an intelligent camera interface that will allow multiple performance and 
communications modes to be supported by the camera system. The objective of this effort being to 
improve the way in which camera systems are deployed by providing a mechanical demonstration 
unit at the completion of the Phase I effort and a conceptual design for an advanced camera interface. 
These objectives will be accomplished through completion of several tasks: 

1 . Design of a passive positioning method for cameras. 

2. Selection of a camera and lens configuration for the system. 

3. Design of a self-contained pan and tilt mechanism for the camera. 

4. Fabrication of the mechanical positioning and orientation system. 

5. Testing of the system for functionality, ease of use, and reliability. 

6. Review of available literature on fish-eye lens pattern processing . 

7. Selection of hardware components for an electro-optical system. 

8. Detailed conceptual design of an electro-optical pan/tilt. 

9. Specification of a smart camera interface. 

1 0. Conceptual design of a smart camera interface. 

1 1 . Preparation of final report. 

These tasks are to be completed on the following schedule: 

Tasks 1, 2, and 6 will be completed 1 month after start of work. 
Task 3 and 9 will be completed 3 months after start of work. 
Tasks 4. 7, and 8 will be completed 4 months after start of work. 
Task 5 and 10 will be completed 5 months after start of work. 
Task 1 1 will be completed 6 months after start of work. 
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The contractor shall provide a Final Report, a working mechanical positioner, a working 
mechanical orientation device, a conceptual design for an electro-optical pan/tilt, and a 
specification and conceptual design for a smart camera interface. 

Task 1 involves locating a satisfactory passive cylindrical shape retention device. There are 
several potential commercial sources and a survey will be conducted to determine the best 
alternative. Task 2 is a similar effort to locate and procure a suitable miniature camera system 
that is conducive to the positioning system and potential combination with the electronics 
development. 

Task 3 will deal with the detailed electromechanical design of a self-contained pan and tilt 
mechanism for the camera selected in Task 2. The output of this effort will be utilized by Task 4 to 
actually fabricate the positioning and orientating devices. The interfaces needed to operate the 
system will also be designed and fabricated during this task. 

Task 5 will involve testing of the system for functionality, ease of use. and reliability while 
using teleoperated manipulators to operate the positioning motions of the camera. 

Task 6 of this project is to review and analyze available literature on vision processing of 
systems equipped with -fish-eye" lenses. Dr. Earnest Hall of the University of Cincinnati has 
completed some studies in this field. The Jet Propulsion Laboratory has performed much work in 
vision systems, including stereo viewing, although thus far. the use of fish-eye optics has not been 
reported by them. These and other sources of information will be reviewed and a bibliography on 

the subject will be developed. 

Selection of potential hardware components for an electro-optical system will be the focus of 
Task 7. Computational requirements and devices with these capabilities will be selected and 
Incorporated into a conceptual design that addresses the potential of a pan and tilt mechanism that 
has no moving parts. Output of Task 8 will be the actual detailed conceptual design of an 
electro-optical pan/tilt. This design will include parts lists and diagrams, but will not be finalized 
and verified in hardware due to cost and time considerations. 

Task 9 deals with the specification of a smart camera interface. It will develop the 
communications protocol and sensory return modes that will provide an advanced camera interface. 
It will also define the auxiliary Interfaces required to make a camera function as an independent 
system (lights, motorized drives, etc.). The objective is to advance the camera interface to a level 
that unburdens other portions of the control system. 

Task 11 will consist of preparation of the final report. All data collected from the testing will 
be included as well as the designs and component listing used. Enhancements to the concept will be 
developed and potential commercial markets will be evaluated. 
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fi. Related RffTfffS r ^ or R&p 

TeleRobotics™ International, Inc.'s principal investigator. Paul E. Satterlee, Jr., has been 
involved in the design, development, and control of manipulators for the past 9 years. His present 
activities are in the development of control systems for mobile robotic platforms and 
servomanipulator systems. This includes the development of software architectures which allow 
manual or automatic path control of vehicles and manipulators. He has also designed numerous 
video interface circuits for special purpose viewing systems. His strong application^ background 
in electronic hardware and software systems will ensure that the designs have full operational 
potential. 

TeleRobotics™ International Inc.'s research associate for this proposal, H. Lee Martin, has 
been a leader of the team developing force-reflecting servomanipulators at the Oak Ridge National 
Laboratory for seven years. While there, he was instrumental in the development of controls and 
mechanisms for servomanipulators. Dr. Martin has recently completed his PhD in the area of 
enhancing manipulation capabilities through the use of digital control methods. He was involved 
with the development of the Advanced ServoManipulator for the Department of Energy from its 
conception through its implementation and has received one patent on the system and has another 
pending. Most recently, he authored a study for NASA through the Oak Ridge National Laboratory 
entitled -Recommendations for the Next-Generation Space TeleRobot System". He has a patent 
pending on the manipulator kinematics and transmission methods of the Man Equivalent TeleRobot. 
(METR). 

Through this extensive hardware experience, the importance of good, reliable viewing has 
become known and respected. Direct experience with passive viewing systems and the difficulty of 
developing full coverage active viewing systems have led to this proposal. Some applications of 
"fish-eye" technology have been performed. One used a hemispherical view to avoid obstacles in the 
guidance of an automated lawn mower (Hall. University of Cincinnati). A second provided for 
intrusion protection in a working robotic environment to replace light fences (H. L. Martin, Y-12 
plant. Oak Ridge, TN). In both of these applications, the -fish-eye" lens allows large viewing 
volumes to be covered by a single camera system with no moving elements. The SURBOT mobile 
robot from REMOTEC Corp. has been outfitted with a -fish-eye" lens and electronics tilting to allow 
navigation with a single camera with no mechanical positioning system (P. E. Satterlee. REMOTEC). 
Each of these implementations have led us to believe that improved methods of viewing techniques 
are possible. Our extensive hands-on experience with remote manipulation indicates that the 
importance of well designed viewing cannot be understated. 
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g. Kev Personnel and Bib l j^™P h y of Directly Related Work 

Two people will perform the majority of the research activity on this effort. They are Paul E. 
Satterlee. Jr.. Principal Investigator, and Dr. H. Lee Martin, Research Associate. Mr. Satterlee 
will perform the electronic design and development of the electro-optical viewing system and the 
smart camera interface. Dr. Martin will develop mechanical designs for the positioning device and 
the mechanical pan/tilt. Vitae of relevant activities follows. 

??v \ f satterlee. Jr. is an electrical engineer. He received his BSEE and MSEE from 
Louisiana State University in 1971 and 1973. respectively. He has completed all PhD coursework 
at the University of Tennessee. In 1984, he was the design leader of the team which won an 
Industrial Research Magazine's IR100 Award for the development of the first all digitally 
controlled servomanipulator system. This award designated this control system as one of the top 
100 technical developments in the country in 1984. He worked at ORNL for 6 years in the field of 
microcomputers applied to real-time control problems before leaving to perform private 
consulting. 

Mr. Satterlee is a cofounder of TeleRobotics™ International, Inc. providing electronic design, 
development, and fabrication expertise. He is also an expert in the language FORTH which has been 
used extensively in the control of servomanipulator systems. Recently, he has been the sole 
provider of electronic and software development support for REMOTEC Corporation's RM-10A 
Manipulator and the SURBOT mobile robot system. He has recently developed an analog version of 
the electro-optical tilting and is in the process of filing patent claims on the device. Mr. Satterlee's 
relevant publications include: 

1 . The State-of-the-Art Model M-2 Maintenance System, 1 984 Topical Meeting on 
Robotics and Remote Handling in Hostile Environments, pp. 147-154. 

2 Distributed Digital Processing for Servomanipulator Control. American Nuclear 

Society Conference. Remote Systems Technology Division. Washington. D. C, Nov. 

1 982 

3. Control Software Architecture and Operating Modes of the Model M-2 Maintenance 

System, 1984 Topical Meeting on Robotics and Remote Handling in Hostile 

Environments, pp.355-366. 
4 Electromechanical Three-Axis Development for Remote Handling in the Hot 

Experimental Facility, American Nuclear Society Conference. Remote Systems 

Technology Division, San Francisco. CA, Dec. 1981. 
5. SURBOT: A Surveillance Robot System for Use in Nuclear Power Plants. Robots Bb 

Conference. Chicago. IL, April 23, 1986. 

6 Mobile Surveillance Robot System, Proceedings of the Amencan Nuclear Society 
Topical Meeting on Robotics and Remote Handling in Hostile Environments. Salt 

Lake City. UT, March 1986. . mi,, 

7 Surveillance Robot for Nuclear Power Plants. Proceedings of the American Nuclear 
Society 33rd Conference on Remote Systems Technology, San Francisco. CA. 
November 1985. 
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8. Analysis and Design Enhancements for the Advanced Servomanipulator, Department 
of Energy, January, 1987. . 

9. Applying the NOVIX Computer to the Real-time Control of Redundant Manipulator 
Systems, Department of Energy, January, 1987. 

nr H l ee Martin is a mechanical engineer with expertise in mechanisms and control. He 
received his BSME from the University of Tennessee graduating number one in his class. He 
obtained a MSME from Purdue University where he developed a patented state-of-charge indicator 
for electric vehicles. He received his PhD from the University of Tennessee in 1986. His 
dissertation topic was "The Reduction of Friction and Inertia Effects in Digitally Controlled Force 
Reflecting Servomanipulators." While an employee of the Oak Ridge National Laboratory he 
developed controls and mechanisms for servomanipulator systems. In 1984 he was a co-winner of 
Industrial Research Magazine's IR100 Award for his contributions to the development of the first 
all digitally controlled servomanipulator system. He was the lead control engineer for the Advanced 
Servomanipulator (ASM) system and has contributed to this work since its inception. 

As a consultant, Dr. Martin aided in the development of the-control system for the REMOTEC 
RM-10A servomanipulator. He has had five patent disclosures at ORNL relating to manipulator 
mechanisms and control strategies and he holds 2 patents. He has also played a key role in technical 
exchange programs with the Japanese. French, and Germans having twice represented ORNL in 
Japan. He founded TeleRobotics™ International. Inc. in January of 1986 to seek opportunities to 
transfer teleoperator technology from the laboratory to the marketplace. 

He has authored or coauthored over twenty papers on the control and use of teleoperated 
manipulators and has edited the book Teleoperated Robotics in Hostile Environments published 
nationally by the Society of Manufacturing Engineers. A few of his relevant publications include: 

1 . An Advanced Remotely Maintainable Force-Reflecting Servomanipulator Concept. 
1984 Topical Meeting on Robotics and Remote Handling in Hostile Environments. 

pp. 407-415. . 

2 Control and Electronic Subsystems for the Advanced Servomanipulator. 1 984 
Topical Meeting on Robotics and Remote Handling in Hostile Environments, pp. 
417-424. 

3 Distributed Digital Processing for Servomanipulator Control, American Nuclear 
Society Conference, Remote Systems Technology Division, Washington. D. C. Nov. 

19 82 - . . . .. .i 

4 Joining Teleoperation with Robotics for Remote Manipulation in Hostile 

Environments, Robotics International, Robots 8 Conference. Detroit, Ml. June 

5. Automatic Camera Tracking for Remote Manipulators.1 984 Topical Meeting on 
Robotics and Remote Handling in Hostile Environments, pp. 383-393. 

6. Advanced Teleoperation in Nuclear Applications. Proceedings of the 1 984 
Computers in Engineering Conference. New York. 1984. pp.302-305. 

7. Duty Cycle Analysis of a Human Controlled Robot. Journal of Robotic Systems. 
December, 1985. 
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8. Recommendations for the Next-Generation Space Telerobot System, Oak Ridge 
National Laboratory Technical Memorandum 9951 , March. 1 986. 

9. Workspace Intrusion Detection Device - Specifications and Implementation, 
Internal report. Oak Ridge National Laboratory. 1 985. 

1 0. Reduction of Friction and Inertia Effects in Digitally Controlled 
Servomanipulators, dissertation for the University of Tennessee, August. 1 986. 

1 1 . Analysis and Design Enhancements for the Advanced Servomanipulator. Department 
of Energy, January, 1987. . 

1 2. Applying the NOVIX Computer to the Real-time Control of Redundant Manipulator 
Systems, Department of Energy, January, 1987. 

f Relationship with F uture Research and R&P 

Good viewing of the worksite is instrumental to successful manipulation either for human 
controlled operations or for computer controlled systems. Many research efforts have been focused 
on the human factors of viewing, color versus black and white, lighting and its relationship to 
viewing, stereo versus monocular viewing, and optimal viewing locations for given task 
configurations. These efforts are typically characterized by extensive testing and incorporate time 
and motion analysis of various tasks performed by a remote operator. The results are often 
task/environment specific and can vary widely from operator to operator, although general trends 
can be developed and supported. Often the superior methods that are recommended by these studies 
are not technically feasible in the actual environments. For example, human factors studies reveal 
that high resolution viewing systems are superior to standard resolution especially when 
inspection tasks are to be accomplished. High transmission bandwidths associated with fixed high 
resolution systems can cause communications problems. The ability to change scanning area 
electronically may offer a technically realizable solution. Human factors analysis reveals that the 
best fixed camera location for a single view is 45 degrees elevated and 45 degrees off the sagital 
plane. These conclusions result from very task dependent testing, but provide a starting point for 
camera base location. A positioning system that is generally located in this area, but is small and 
flexible enough to be placed throughout the worksite offers an exciting solution. The self-contained 
pan/tilt system could even be deployed like another tool and attached via a clamp and umbilical to a 
desirable point on the worksite. The viewing potentials are unlimited. We like to envision the 
operation much in the same way that a photographer sets up his studio for the best portrait and then 
only needs to aim and focus the camera during the actual working session. 

The most important aspect of advanced research that this proposal relates to is the 
development of 7 degree of freedom manipulators (redundant kinematics). These new manipulators 
will place a tremendous need for flexible viewing techniques. Now these manipulators can work 
around and behind objects, but their efficiency will be diminished if the ability to see the objects .s 
not also provided. More important than the force reflection capabilities of these manipulators is 
the ability of the operator or computer control system to properly see the worksite. The passrve 
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flexible positioning system proposed is the safest and simplest way to provide complete flexibility 
for viewing the motions of future redundant kinematic manipulators. The method is "safe" because 
it does not rely on rigid members to hold the camera (i.e., the flexible fixture will comply easily 
should it collide with the worksite or a segment of the manipulator). The method is simple because 
no active drives are needed to position the camera. A camera positioning system with equal 
positional coverage would require at least four active drives and linkages, and their associated 
controls, power systems, decreased reliability, and bulk. Prior experience has revealed that these 
active, structurally intensive systems have always had a limited ability to operate in the near 
vicinity of the manipulators. In fact, due to collision considerations, these systems are often 
designed to stay out of the motion range of the viewed manipulator system. This limits the quality of 
the view produced and renders these viewing methods less than optimal. 

Other researchers are actively developing end-effector viewing techniques. These systems 
utilize a wrist mounted camera to locate and track objects of interest at the end-effector. They are 
very useful for automatic target attachment, but the continuous motion of such a camera on an 
active arm makes human viewing difficult. If the view is stabilized by holding the arm motionless, 
the result is that the manipulator is reduced to a camera positioning system, requiring extra 
manipulators for multiple armed tasks. The flexible positioner in concert with the self-contained 
pan/tilt seeks to address this problem by allowing the manipulator to position the camera in an 
ideal spot for a given task and then use direct operator controls for pan and tilt to provide the 
viewing angles in the worksite while the task is being performed. Some may question if the need to 
reposition the camera frequently might interfere with the task progress and efficiency. It has been 
our observation that when tasks are being performed, the first activity that a human operator 
performs is to position the cameras so that the worksite can be properly viewed. The operator then 
proceeds to perform the task and the cameras are seldomly repositioned, although they are 
frequently reoriented. For this reason, the incorporation of pan and tilt within the camera system 
will allow uninterrupted task activities to be performed. Even for robotics tasks that utilize 
end-effector sensing, the second view in close proximity to the worksite can give useful depth 
information or a second verification of operations to the autonomous controller. 

The development of a method for using a "fish-eye" viewing system will provide a more 
reliable and more versatile system to be implemented. Not only will there be no moving parts 
associated with the concept, but the zoom function can also be incorporated into the device. As the 
technology for charge coupled device (CCD) array cameras improves and the resolution increases 
(it has moved from 32x32 to 1024x1024 in the past 7 years and is headed for 2048x2048), the 
methods developed in this proposal will take advantage of this future technology. 

Additionally, if the cameras were fixed to the transport base, multiple view triangulation to 
determine position would be more accurate since mechanical nonlinearities which accompany drive 
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mechanisms (backlash, encoder resolution, compliance) are eliminated. A stereo viewing system 
with no moving parts based on the concept to be developed would also be an area for follow-on 
research and development. 

q. Facilities 

TeleRobotics™ International. Inc. is a new company developing human controlled 
electromechanical systems for remote applications. This company was formed specifically to 
provide creative energy in the application of robotics to hazardous environments including 
mechanisms, controls, the human interface, and auxiliary devices. We have been successful in 
attracting and completing several studies and prototype developments that will positively affect the 
way in which remote manipulation is accomplished in the future. The effort detailed here will rely 
heavily on the ideas and past experiences of the two investigators in this field. Tele Robotics™ 
International. Inc. has a 350 square foot laboratory facility supported by a VME bus computer 
supplied with a Motorola 68000 and I/O devices. Additionally. ? TeleRobotics™ International, Inc. 
has several stand alone development and design workstations that may be useful in this activity. 
The hardware required to perform this effort is minimal except for the specific components listed 
in the budget for this activity. Any additional equipment cost will be born by TeleRobotics™ 
International, Inc. 

h. Consultants 

Mechanical detailing will be provided by consulting services. Our historical rate for this 
service is $30/hour. Approximately 200 hours are estimated for the detailing of the 
self-contained pan/tilt. We also utilize local machine shop facilities in the Oak Ridge area to 
fabricate our prototype equipment. We have had previous hardware experience and are confident 
that the estimates described in this proposal are reasonable and feasible for the prototype 
self-contained pan and tilt system to be developed. 



1- Potentia l Applications 

Potential NASA applications are numerous. In addition to application with the dexterous 
telerobot being developed for space station construction, this type of imaging system could be used 
for docking operations on future satellites, path display on unmanned vehicles, and inspection of 
cluttered structures where conventional cameras could not reach. Since the unit will be smaller, 
lighter, and less expensive than Its mechanical counterpart, the viable applications are many. 
Other government applications include remote environments for hazardous material handling 
(Department of Energy, Department of Defense). 
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The commercial potential for the small, self-contained unit is also significant. Almost every 
-quick market" store and bank have a camera on a pan and tilt logging scans of the facility for 
security purposes. The device described in this proposal would be less expensive and more 
versatile than the conventional equipment now in place. This opens a large potential market for the 
device in surveillance applications. Mobile vehicles that are remotely controlled would also benefit 
from the development of such a device. If the advent of an autonomous automobile occurs, this type 
of viewing attached to a computer imaging system would provide an excellent sensory input device. 
The preprocessing features would allow the availability of high resolution without the processing of 
areas of information not of interest at that particular point in time. 

a. Related Proposals or Awards 

TeleRobotics™ International. Inc. has been awarded a Phase II contract from the Department of 
Energy. Washington, D. C. 20545 in the area of development and fabrication of an advanced 
teleoperated manipulator with redundant degrees of freedom.,- The potential development of an 
advanced viewing positioning system would complement this activity greatly although there is no 
directly related funded item in any other activity that TeleRobotics™ International. Inc. is involved. 
TeleRobotics™ International. Inc. is presently completing a NASA Phase I effort aimed at improving 
dual arm manipulation control techniques. During this effort, the importance of good viewing has 
been reemphasized, especially when two manipulators are used in close proximity to each other. 
While these efforts reveal a synergistic need for the device that we are describing herein, no 
related efforts are ongoing or proposed in this area. 
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APPENDIX C - SBIR PROPOSAL SUMMARY BUDGET 
(INSTRUCTIONS ON REVERSE SIDE) 

NATIONAL AERONAUTICS AND SPACE ADMINISTRATION 

SBIR . SOLICITATION 



FIRM: 

TeleRobotics International, Inc. 

PRINCIPAL INVESTIGATOR: 

Paul E. Satterlee, Jr. 

(See Instructions on Back of Form) 
MATERIAL: * . 

a. Katerials and Supplies - $5 » 500. 00 

b. Publication costs - $1,000.00 

c. Consultant Services - $6,000.00 

d. Computer Services - $0.00 

e. Subcontracts - $0.00 



PROPOSAL NUMBER: 



07.01 5600 



f . Other - $0.00 



*See Attatched Sheet 



PERSONNEL: 

PI - PES 
Engr - HI*: 



600 hrs 
150 hrs 



$24/hr 
$24/hr 



$14,400 
$ 3,600 



TOTAL PRICE 

12,500.00 



$18,000.00 



$ $ 1,600.00 

OTHER DIRECT COSTS:* 

Travel - $1,600.00 

*See Attatched Sheet - 



OVERHEAD: 

. (55% x $32,100) 



$ $17,655-00 



■ " ' $ $ 0.00 

GENERAL AND ADMINISTRATIVE (G&A): 



PROFIT: 



$ $ 



0.00 



TOTAL PRICE PROPOSED $ 



$49,755-00 



TYPED NAME AND TITLE: 



H. Lee Kartin, President 



SIGNATURE: 



THIS PROPOSAL IS SUBMITTED IN RESPONSE TO NASA SBIR PROGRAM SOLICITATION 
36-1 AND REFLECTS OUR BEST ESTIMATES AS OF THIS DATE. 



C 3 

DATE SUBMITTED 



PROPOSAL PAGE NO. 17 



APPENDIX C 
SR1R Propos al Summary Budget 



, Material 

a. Materials and Supplies = $5500.00 

Camera- $800.00 

Drive motors/reducers-2 @ $500 each 
Drive amplifiers-2 @ $350 each 
Mechanical fabrication of parts-$3000 

b. Publication costs = $1000.00 

50 pages @ $20.00/page 

c. Consultant Services = $6000.00 

System development and design check, $30.00/hour for 200 hours 

d. Computer Services = $0.00 

e. Subcontracts = $0.00 ? 

f. Other = $0.00 



3. Other Direct Costs 

Travel -$1600.00 
It is anticipated that two trips will be required to properly execute 
this activity, one at the beginning for task definition and one at the 
conclusion to present the results of the effort. Each trip will require 
two participants and the cost is estimated at $800.00 per trip. 



ATTACHMENT A 
NATIONAL AERONAUTICS AND SPACE ADMINISTRATION 
SBIR SOLICITATIONCJ-1 
PHASE II PROPOSAL COVTO SHEET 



PROPOSAL NUMBER ( 

Solicitation Four Digit Last Four Digits or 

Number Subtopic No. Firm Telephone No, 



^ Numpe 
t-Enterr€r 



II n7 oi 5fino — — . p • 

iginal Phase I Proposal No. Here & on Attach . B 

PROJECT TITLE: 
Electro-Optical Pan/Tilt/Zoom: A Miniature Viewing System 
SUBMITTED BY: FIRM NAME: TeleRobotics International , Inc " 

MAIL ADDRESS: , . . . 

8410 Oak Ridge Highway 

CITY, STATE , ZIP: Tenness ^ 3 7931 

1. The above organization certifies that it is a small YES: xx NO: 
business meeting the definition in Section 2.2 of 

NASA SBIR 87-1 and U.S.C. 631, and other eligibility 
requirements of Section 1.3 of NASA SBIR 87-1. 

2. The above organization certifies that a minimum of YES: xx NO: 
fifty-one percent of any Phase II contract awarded 

under this solicitation will be carried out within 
the firm. 

3. The above organization has included in this proposal YES: X x NO: 
a commitment from a non-Federal source for follow-on 

funding for "Phase III." 

ENDORSEMENTS 

Pincipal Investigator: Corporate/Business Official: 

T?j?r. ° r - «• Lee Martin TitK! ° r - h - Lee Martin 

H tle ' President Phone: President _ 

Phone: {615) ^q^soq . rnone. { ^ 5) r690 .^ 00 



Signature of pYl. Signature ot otticiai 

PROPRIETARY NOTICE (If applicable, see Section 6.5 of NASA SBIR 87-1 

NOTICE: The information (data) on pages 3-17 of this proposal 
constitute a trade secret and/or information that is commercial or 
financial and confidential or privileged. It is furnished to the 
Government in . confidence with the understanding that it will not, 
without the permission of the offeror, be used or disclosed other than 
for evaluation purposes; provided, however, in the event a contract is 
awarded on this proposal, the Government may obtain additional rignts 
to use and disclose this information (data). 




ATTACHMENT B 
NATIONAL AERONAUTICS AND SPACE ADMINISTRATION 



PHASE II PROPOSAL SUMMARY 



TO BE COMPLETED BY OFFEROR 



PROPOSAL NUMBER 



sbii C3 1 " 11 o7 * oi " 56o ° 

NAME AND ADDRESS OF OFFEROR 

TeleRobotics International, Inc. 
8410 Oak Ridge Highway 
Knoxville, TN 37931 

PRINCIPAL INVESTIGATOR 

Dr. H. Lee Martin 

TITLE OF PROJECT 
Electro-Optical Pan/Tilt/Zoom: A Miniature Viewing System 

TECHNICAL ABblKM.il LIMIT 200 WORUS TTNCLrTfATEMENT ON ^ 

A necessary exponent of teleoperated robotic manipulation » J^^^;. 
technological advances in remotely controlled v ^ on ^IZtllls TA needed to enhance the 
remote manipulators. The development of improved v"™? ln t L Phase I research 

overall performance of remotely operated tele robotic man ^^J"^ Lomplishes 
effort, a miniature omni-directional camera prototype was than reorienting the 

remote viewing by simply moving the optics of the camera system ^the* fan r feasibility 
entire camera. Considerable reductions in size and weight were realize to convert 

demonstration. A theoretical analysis was performed to develop » 9 di ital electronics 

a fisheye image into an undistorted object plane "P" 5 ^^^"^^^^ with a fixed camera 
The resulting algorithm allows pan/tilt/zoom ^^J^.f ."^i^^ was also created, 
and was validated experimentally in Phase I. An el ect r ° n ^ p h ^^ n ^tion of an electro- 

The primary goal of the Phase II effort is the hardware ^£J n ££™ ch of Phase j 
optical pa-./tilt/zoom camera system for remote viewing. The initial researcn 
^vided F th'e ..sight and theoretical background to -^ofiJn s^ns^ng devi es nereis the 
approach. The elimination of mechanical components and motion sensing ^i elera ents 
reliability while significantly reducing size, weight, and maintenance costs y 
to future NASA implementation of telerobotic viewing systems. 

private sector, surveillance and security systems that presently use »«"xp 
mechanical orientation and zooming methods can be replaced by this technology with 

savings and performance improvements. 

"FOR NASA USE ONLY 



AWARD NUMBER 



Electro -Optical Camera System 



3. Results of the Phase I Work. 

Remote manipulation incorporates many technologies to accomplish 
maintenance and assembly tasks without the direct presence of human 
beings Other than the manipulator itself, advantageous viewing of the 
worksite is the most critical element for efficient teleoperation and 
verification of robotic task completion success. As machine vision 
techniques improve, worksite viewing is becoming increasingly 
important for autonomous manipulator operations in dynamic 
environments. The thrust of the Phase I effort was to explore and 
develop improved methods for placement and orientation of the camera 
system. 

The major technical goals stated for the Phase I effort were: 
1. Hardware realization of a passive camera positioning system 
with complete worksite coverage potential. 

2 Hardware realization of an active, self-contained camera 
orientation device based on mechanical movement of the camera 
optics. 

3 Conceptual design development of a hemispherical viewing 
method based on electro-optics with no mechanical mechanisms 
to accomplish pan, tilt, and zoom. 

4. Conceptual design development of a smart camera interface that 
provides the ability to control, communicate, and add cameras 
to a working system. 
In the six month Phase I period, the following major milestones 
were accomplished: 

1. Design and fabrication of a passive camera positioning system 
(see Figure 1 concept and Figure 2 implementation). 

2. Design and fabrication of an active, self-contained camera 
orientation device based on mechanical movement of the camera 
optics (see Figure 1 concept and Figure 2 implementation). 

3 Development, validation and testing of an algorithm that creates 
an undistorted image from a "fisheye" image (see Figure 3 
"fisheye" distorted image and Figure 4 undistorted planer 
section). 

4 Conceptual design and feasibility analysis of a camera hardware 
system that undistorts the "fisheye" image in real-time (see 
Figure 5). 
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ATTACHMENT C 

FORM APPROVED 
OM8 NO. 

3090-0116 



TeleRobotics International, Inc. 
8410 Oak Ridge Highway 
Knoxville, TN 37931 



✓ PC OF CONTRACT (Chec*) ~__ 

]ffp Qcpff Qcpif 

] Fp| n ° ther < specify> 



[~] CPAF 



Dr. H. Lee Martin, Pres, 



4. TYPE OF CONTRACT ACTION (Check) 



X 


A NEW CONTRACT 




0 LETTER CONTRACT 




B CHANGE ORDER 




E. UNPRICED ORDER 




C. PRICE REVISION/ 
REDETERMINATION 




F. OTHER (Specify) 




6 PROPOSED COST (A+B*C) 



A. COST 

$ 463,805 j$ 32,466 



$ 496,271 



ZaCE(S) AND PERIOD(S) OF PERFORMANi.* 

■ Knoxville, Tennessee; 24 month ARO 



E. REF. 



. LINE ITEM NO. 



B. IDENTIFICATION 



Design, development and delivery of Electro- 
Optical pan/tilt/zoom miniature viewing system 



Attached 



CONTRACT ADMINISTRATION OFFICE 



c ppnv.HF NAME. AD DRESS. AND TELEP HONE NUMBER FOR THE FOL LOWIN G Of o ^o^ 

^ — ■ r B. AUDIT OFFICE 




171 VES I~]n0 

<ir<Y™"ide£ify item(*U ciutomer(s) end centred number<*)> 



Qyes Qno 



11A DO YOU REQUIRE GOVERN* 

11 Sent contract financing 

TO PERFORM THIS PROPOSED 
CONTRACT? (If 'Yes, complete 
Item I IB) 



{^] YES 



11B. TYPE OF FINANCING U one) 



j — | ADVANCE^ 



E PROGRESS 
PAYMENTS 



PAYMENTS 
|~| GUARANTEED LOANS 



FAR PART 31 COST PRINCIPLES? (If "No." txplam) 



[xjYES QnO 




□ yes Qno 



ACCOUNTING STANOAROS? W " Ye.." explain in proposal) 

D y ES Q NO . — 1 ' I* , ' rt r r«i rr11 rir . ^ ^.;^..»c »nri/or actual costs as o< this date. 

Thisproposafssub minedioresPonsetotheRFP contrac t mod,<.cafon, etc. .n It em l and rejects best ^ 

15, NAME AND TITLE (Type) 



□ YES [jpNO 



Lee Martin, President 



16. NAME OF FIRM 

TeleRobotics International, Inc. 




1 A 1 » .1 IM 



18. DATE OF SUBMISSION 



£ 3 



STANDARD FORM 1411 (1X 63) 



Hectro-OpufcaK>mera System 



4 



^ ,tom Cn.t Rreakdown- FI FC TRQ-QPTIQAL PAN^TILT/ZOOM 
CAMERA SYSTEM - Tplaj, g_il£l ($ 4 96,271 ) 

nir^ct Material ($97.120) 
Subcontracted Items 

$20,000 (Modifications to camera mechanical 
configurations, printed circuit board fabrication) 

Standard Commercial Items 

$77,120 (see attached list of items) 

Raw Material (None) 
Direct Labor ($205.400) \- 

PI-HLM (hours-2000, rate-$26/hr, total-$52,000) 

SDZ (hours-3400, rate-$21/hr, total-$71 ,400) 

PES (hours-2000, rate-$26/hr, total-$52,000) ^tfS 

TRI support staff (hours-1500, rate-$20/hr/ total-$30,000) 
lnd^r^Co^t^L$4^^iai /2J» Z*o 

Labor overhead and general and administrative overhead last two 

year's average: 65% 
nthor Cngts ($27.775) 

Travel 

4 update trips, two people each, one night tf> 
(8*(400-airfare+75-hotel+25-ground trans.@) per diem)) 
= $4489" 44ogo 
Shr^rt Courses 

Efficient preparation for DSP development (2@$1500) 

Report Preparation 

Reproduction costs for reports ($375) 

Consultants 

Don Bouldin, Dan Newport for ASIC chip developments 
($20,000) \[\%« 
Pr^fiyc ^ntinaencv (7% Qf tola! = 

Royalties <$JLM1 
Facilities Capital gosjoj. Mojiex 
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Materials Support Information 

The items listed below are the major items required for the 
development of an electro-optical pan/tilt/zoom camera system 
described in the accompanying proposal. While the effort has many 
parts the single line item deliverable is the camera system described. 
Where available, cost estimates have been obtained from data sheets or 
phone conversations with vendors. Engineering judgement has been 
used based on prior experience to estimate the cost of fabrication 
activities since no detailed design is presently available for obtaining 
price quotations. 



Subcontracted material items: ($20,000) 

Camera packaging modification 2@$5000 

Multilayer printed circuit board fabrication 5 @ $2000 



Total 

Standard Commercial 



$20,000 



Items: ($77,120) 



Fisheye Lens 


Nikor 8 mm f/2.8 180° 


1865.00 


2 


VRAM 


TC524256Z-10 1 Mbit 


75.00 


32 


Video Control 


TMS34061-12 


40.00 


2 


DRAM Memory 


64K Bytes 35 nS 


50.00 


2 


SRAM Memory 


64K Bytes 35 nS 


50.00 


2 


Dual-Port RAM 


4K Bytes 70 nS 


125.00 


2 


Microprocessor 


N80C196 Intel 


40.00 


4 


Misc. Logic 


F series logic 


3000.00 


1 


DSP 


DSP56001 


300.00 


4 


DSP 


Analog Devices ADSP2100 


300.00 


4 


Flash A/D 


AD9012 


100.00 


4 


MM I Platform 


Mac II 


6000.00 


2 


Digitizer 


256 gray scales 


6000.00 


1 


Software 


Programming tools 


2000.00 


1 


Monitor 


HDTV resolution 


5000.00 


2 


PAL Programmer 




1500.00 


1 


Camera 


CCD 


10000.00 


2 



TeleRobotics International, Inc. 
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DSP Development System 5000.00 1 
1 00 MHz Scope . 8000X0 1 

TOTAL 77,120.00 



TeleRobotics International, Inc. 
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10. Representations, Certifications, and Other 
Statements of Offerors 

Required representations, certifications, and other statements of 
the offer are attached per the SBIR Program Proposal instructions. 

11. Follow-on Funding Commitment for Phase III 

A venture group out of Clearwater, Florida has been contacted 
concerning the progress and promise of our Phase I efforts^ L I.S 
Enterprises, Inc. is a firm with real-estate and high tech holdings 
throughout Florida and now expanding into the Southeast. They have 
substantial backing ($90 million in assets) and have an interest in the 
omni-directional viewing system for commercial surve.l lance 
applications. Our conversations have been very fruitful thus far. A 
letter of commitment from U.S. Enterprises follows. 

TeleRobotics International, Inc. also has interests in the 
commercialization of the fisheye electro-opticah imaging system for 
remotely operated vehicles. We plan to develop algonthms that will 
support autonomous navigation via image analysis to further the 
commercial acceptance of this technology. 



TeleRobotics International, Inc. 
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NATIONAL AFRONAUTICS AND SPACE ADMINISTRATION 

sbirC "3-n 

REPRESENTATIONS AND INSTRUCTIONS 



APPBPP.P. TBT .FWwnTTCS INTERNATIONAL , INC. 



1 52.203-2 CERTIFICATE. OF INDEPENDENT PRICE DETERMINATION 

(a) IS) Jl^lnttli'Sffir have been arrived at independently, without, 
for tie purpose of restricting competition, any c? n J^ u ^;, c S , "^» ° P 
agreement with any other offeror or competitor relating to O) those P"ces, 
tii)Tne inientU to submit an offer, or (iii) the methods or factors useo to 

" 1CUl (2) e Te 'prices lOToffer have not been and will not be knowingly dis- 
closed by the offeror, directly or indirectly, to any other offeror or competi- 
to? before bid opening (in the case of a sealed bid solicitation) or contract 
award (in the ca?e of a negotiated solicitation) unless otherwise required by 

l3W; (3) No attempt has been made or will be made by the offeror to induce any 
other ( conce?n to Tubmi? or not to submit an offer for the purpose of restricting 

(bTllc^signature on the offer is considered to be a certification by the 
signatory JJ^J!^^; offeror . s organization responsible for determin- 
ina the prces being offered in this bid or proposal , and that the signatory has 
not Jartici-pated and will not participate in any action contrary to subpara- 
graphs (a)(1) through (a)(3) above; or t ha fni 

(2) i Has been authorized, in writing, to act as agent for the fol- 
lowlna principals in certifying that those principals have not Participated, and 
iTv I not parlici pate in any action contrary to subparagraphs (a)(1) through 

SinKl? nam^f p£S(«n T ^ or 
determining the prices offered in this bid or proposal and the title of his or 

her position in the offeror's organization); n . flp . . k nam(jd 

P (ii) As an authorized agent does certify that the principals named 

in subdivision (b)(2)(D above have not participated, and wi 1 not participate 
n any action conirVy to subparagraphs (a)(1) through (a) 3) above; and 

(iii) As an agent, has not personally participated, and will not par- 
ticipate, n any action ?ont ary to subparagraphs (a)(1) throug « (a 3) above 
(c) If the offeror deletes or modifies subparagraph (a)(2) above, the offeror 
lust furnish with its offer a signed statement setting forth in detail the cir- 
cumstances of the disclosure. 



(End of Provision) 




2 52.203-4 CONTINGENT FEE REPRESENTATION AND AGREEMENT | 

(a) Representation. The offeror represents that, except for full-time bona 
fide employees working solely for the offeror, the offeror - 

(NOTE- The offeror must check the appropriate boxes, ^or interpretation of the 
representation, including the term "bona fide employee", see Subpart 3.4 
or the Federal Acquisition Regulation.) 

(1) ( ) has, (X) has not » employed or retained any person or company to 
solicit or obtain this contract; and 

(2) ( ) has, (X) has not, paid or agreed to pay to any person or company 
employed or retained to solicit or obtain this contract any commission, percent- 
age, brokerage, or other fee contingent upon or resulting from the award of this 

( b) tr Agreement . The offeror agrees to provide information relating to the above 
Representation as requested by the Contracting Officer and, when. subparagraph 
(a)(1) or (a)(2) is answered affirmatively, to promptly submit to the Contract- 
ing Officer - Qmpleted $tandard Form U9) statement of Contingent or Other Fees, 

(SF 119) • or 

(2)' A signed statement indicating that the SF 119 was previously submitted 
to the same contracting office, including the date and applicable solicitation 
or contract number, and' representing that th« prior SF 119 applies to this offer 
or quotation. 

(End of Provision) 
3 18-52 203-70 CONTRACTS BE TWEEN NASA AND FORMER NASA EMPLOYEES 

j~ — -^( nasaTfaTsuppl ement ) 

The offeror represents that he or she ( ) is, or \yh is not, an 
individual who was employed by NASA during the past two (2) years, Or a firm in 
which such former employee is a partner, principal officer majority sharehold- 
er, or which is otherwise controlled or predominantly staffed by such former 
employees. 

(End of Provision) 
4. 52.215-6 TYPE OF BUSINESS ORGANIZATION ^ 

The offeror or quoter, by checking the applicable box, represents that - 

(a) It operates as (X) a corporation incorporated under the laws of the State 
of T^essFE » ( ) an individual, ( ) a partnership, ( ) a nonprofit 
organization, or ( ) a joint venture; or 

(b) If the offeror or quoter is a foreign entity, it operates as ( ) an indi- 
vidual, ( ) a partnership, ( ) a nonprofit organization, ( ) a joint venture, 
or ( ) a corporation, registered for business in _ . • 

country 

(End of Provision) 



^—3 



5 52.215-11 AUTHORIZED NEGOTIATORS 

The offeror or quoter represents that the following persons are 
authorized to n^otf/te n itAehalf w?th 

this request for proposals or quotations: (list names, titles, ana i-e.ep 
numbers of the authorized negotiator^. _ ^ 



6 52.215-20 



(End of Provision) 
PLACE OF PERFORMANCE £ 



f»l The offeror or quoter, in the performance of any contract resulting from 
it- i^VtttiL t ) intends (V) does not, intend check applicable block), 
this solicitation, ( ) intenos, j x; « U J • different address from the 

insert in the spaces provided below the required information. 



Place of Performance 
(Street Address, City, 
County, State, Zip Code) 



Name and Address of Owner and 
Operator cf the Plant or Facility 
if Other than Offeror or Quoter 



(End of Provision) 
52.219-1 fMfM Ril^TNFSS CONCERN REPRESENTATION 




The offeror represents and certifiers part of its offer that it 
en^sVbe^ni^ 

in this provision, means a concern, including Us af ^^l.^Vwh it 
entlv owned and operated, not dominant in the field of operation n whuh u 
bidding Tn Government contracts, and qualified as a small business under the 
size standards in this solicitation. 



is 



8 52.219-2 SMALL PIS 



(End of Provision) 
DISADVANTAGED BUSINESS CONCERN REPRESENTATION 



(a) Representation. The offeror represents that it ( ) is, i%) is not, a 
small disadvantaged business concern. 

<b) SAsian^lndian American," as used in this Provision means 
citizen whose origins are in India, Pakistan, or Bangladesh. 



a United States 



"Asian-Pacific American," as used in this provision, means a United States 

H ^^^^^^^o^ means African Indians, 

^Ln'l^'nS! cfnceTn^as'used in this provision, TO ans a concern, in- 
-Small *"JJ"*!Lf "g^* is independently owned and operated, not dominant 
1 ° e'f d 3 e a lion which it is bidding on Government contracts and 
JSallfled as a small business under the criteria and size standards in 1, CFR 

121 ' "Small disadvantaged business concern," as used in this Provision means a 
sma i uisauva »s least 51 Dercent owned by one or more 

uals found to be qualified by the SBA under 13 CFR 124.1. 

(End of Provision) 
9 52.219-3 WOMEN-OWNED SMALL BUSINESS REPRESENTATION^' ^ 

(a) Representation. The offeror represents that it ( ) is, <X> 1* not, a 
women-owned small business concern. 

121 ' "Women-owned", as used in this provision means a ^{^^SVhSHlli * 
least 51 percent owned by a woman or women who are U.S. citizens ana wno 
control and operate the business. 

(End of Provision) 

10 52 220-1 PREFERENCE FOR LABOR SURPLUS AREA CONCERNS^ — 

award in case of tie offers ev * determine whether the 

American Act clause of this solicitation, in oroe u offeror must 

to more thanJO percent of the contract price. 



(b) Failure to identify the locations as specked above will f«£f c c °Ztt 

with the obligations of an LS4 concern. 

(End of Provision) 
11 52.222-21 CERTIFICATION OF NONSEGRE GATED FACILITIES ^_ 

(a) -Segregated facilities", as used in this provision means any waiting 
rooms wo^k areas, rest rooms and wash rooms, restaurants and other eating 
*rT*V time clocks locker rooms and other storage or dressing areas, parking _ 
?nt1 dHnkina fountains! recreation or entertainment areas, transportation, and 

n^^ned!^^^ «. 

ratifications from proposed subcontractors for specific time periods) it will - 

m Obtain identical certifications from proposed subcontractors before 
the «l"d of suEcont?acU under which the subcontractor will be subject to the 

Equal Opportunity clause; . 

m Rptain the certifications in the files; and 

• 3 Forward the following notice to the proposed subcontractors (except f 
the proposed XnEr actors have submitted identical certif ications for specif ic 
time periods): 

NOTICE TO PRO SPECTIVE SUBCONTRACTORS OF REQUIREMENT FOR 
~ CERTIFICATIONS OF N0NSEGRE6ATED FACILITIES 

A Certification of Nonsegregated Facilities must be 
submitted before the award of a subcontract under which the 
subcontractor will be subject to the Equal Opportunity 
clause. The certification may be submitted either for each 
subcontract or for all subcontracts during a period (i.e., 
quarterly, semiannually, or annually). 

NOTE: The penalty for making false statements in offers is prescribed in 18 
U.S.C 1001. 

(End of Provision) 
12 52.222-22 PRBHOUS CONTRACTS AND COMPLIANCE REPORTS 

<a) It (X) has he ( 0f > e h°as noCpar'ucip'fted in a previous contract or subcon- 
Ir'act sublet either to the I'J Opportunity clause of this solicitation, the 




developed and has on 



Cause original!, contained in Section 310 of Executive Order Ko. 10925, or the 
!T?t C <&Tas '< W-T ni-SMS c^p"ance reports; and 

(End of Provision) 

13 52.222-25 AFFIRMATIVE ACTION COMPLIANCE f^~~ 

The offeror represents that (a) it (X) has _ ... . . 

file ( ) has not Sevel oped and does not have on frte, at each establishment, 
I })Vrltile Action programs required by the rules and regu atio of t e 

con£^ 

rules and regulations of the Secretary of Labor. 

(End of Provision) 

14 USE OF GOVERNMENT-OWNED PROPERTY (45-200^ 

j / \ ^««c nnt ( V * intend to use in any contract 
^rd T e h d%f rrrJrof'this'so^ciralionAny ^ni-owned U, facilities, 
(2) special test equipment or (3) special tooling. 

by the offeror as a part of the proposal: 

(1) Identification and quantity of each item and the value thereof. 

(2) Identification of the Government contract under which acquired and 
written permission for its use from the cognizant Contracting Officer. 

(3) Rental provisions. 

,A^ A*+ a n f thP last review by the Government of its property control 
and accounl^syst^ and describees tafcen to correct any deficiencies 



found. 



(5) A statement that the offeror has reviewed, understands and can comply 
with 11 property management and accounting procedures in the solicitation FAR 
Subpart 45.5, and NASA/FAR Supplement. Subparts 18-45.5, 18-45. /u, ana w 

(6) A statement indicating whether or not the costs associated with 
subparagraph (5) above are included in its cost proposal. 

(7) The dates during which the property will be available for use, and if 
used in two or more contracts, the amounts of respective uses in sufficient 
detail to support proration of the rent. 

(End of Text) 



15 RELATED PROPOSALS AND AWARDS REPRESENTATION 

The offeror, by.checking the JPPllcjM. *x .<*™«£ 1?£ en'anVuLt 

(End of Text) 
(END OF PRESENTATIONS) 



SIGNATURE BLOCK 



By sl.n.tyr, below ^ f offeror certjn^ t.at an ^ 
U.S.C. 1001 for making false statements in proposals. 

s 




Signat^e 

Harry Lee Martin^ 
Typed Name 

Title 
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Electro-Optical Camera System 



5 Parts location and selection of the major electronic elements 
that will comprise the hardware realization of the electro- 
optical pan/tilt/zoom camera system. 
In the past, large pan and tilt units and bulky cameras have been the 
only source of viewing feedback from the remote site. These camera 
orientation and positioning systems have often been as complicated as 
the manipulators themselves. A miniaturized alternative, shown 
conceptually in Figure 1 and prototyped in the Phase I effort, is a self- 
contained camera that accomplishes remote, viewing orientation by 
simply moving the optics of the camera system, rather than reorienting 
the entire camera. The savings in subtended volume for camera 
orientation throughout the range of viewing motion is significant, 
reducing from 4 cubic feet for typical systems to 1/4 of a cubic foot 
for the prototype system. The reduction in size also results in a 
reduction in weight allowing the entire viewing package including the 
positioning system to weigh less than 20 pounds. A mechanical 
prototype of the concept was fabricated in Phase I and is shown in 
Figure 2. The mechanical-optical pan and tilt of the unit is 
accomplished by a lens mounted mirror. The entire package is then 
placed on a counterbalanced four-bar linkage that allows the camera to 
be located in the worksite by using one of the manipulators. This 
combination of simplicity and flexibility offers a viable alternative to 
approaches that use fixed location camera systems or complex 
articulated arms that crowd the worksite and interfere with the 
manipulation systems. 

The second major activity of the Phase I research was to further 
reduce the size and complexity of the camera system by accomplishing 
the pan, tilt, and zoom capabilities without moving parts. The 
conceptual design of an innovative method of remote viewing which 
uses state-of-the-art electronic components was developed. The new 
concept uses a fisheye lens and high resolution digital camera to 
control all camera functions including electronic pan, tilt, and zoom 
capabilities. A complete hemispherical field of view is provided 
without any mechanical movement of the camera and can be undistorted 
by the electronics that operate the camera. 

The Charge Coupled Diode (CCD) camera makes use of a new high 
resolution CCD array image sensor recently developed by Toshiba Corp. 
The new image sensor provides an order of magnitude increase in the 
number of image elements integrated on a single chip. An electro- 
optical camera was conceptually designed and the transformation 
algorithms developed that allow the new CCD sensor and a "fisheye" 
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Figure z - m«.»-~- — d oom 

„ m achieve electro-optical pan tilt, ana 
lens equipped camera to acn eve ^ ^ ^ 

over a complete 2k steradian hem P fisheye irna ge 

Utilizing digital image J-™-*! projection into an 

^r^;°pian a e Y $~fi^z t x » em? 

using* a Macintosh II and the C , pro^mmg 'a = tne Joretical 
computer implementation of he algorrth m g ^ ^ com plete 
analysis as can be seen in Wj^, J ^ basis for testing he 
2, steradian fisheye image that was ^ Qf apply , ng the 

algorithm. The view shown , «. *9 U £ 4 , axis view direction, 
algorithm using the normal view or para jmp , emente d and 

Not only was *e algorithm dev eloped bu t it re 
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overhead. Our analysis reveals that the utilization of a digital signal 
processor (DSP) which is solely designed for the implementation of 
this algorithm will provide significant (1-2 orders of magnitude) speed 
increases. By performing the transformation once to create a look-up 
table (in less than one second), the subsequent image transformations 
can be accomplished at the 30 frames per second rate. The electronic 
methodology needed to implement this concept in real-time has 




Figure 3 - Scanned fisheye image prior to de-skewing. 
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Figure 4 - Scanned fisheye image after application of algorithm. 



been researched, availability confirmed, and several concept diagrams 
of the design have been developed for Phase II development and 
application. 

Since the Phase II proposed effort builds on the results of our 
algorithm development from the Phase I effort that undistorts the 
hemispherical image into a planer image at the orientation and zoom 
desired, the essence of the theoretical analysis that was fully 
developed and presented in the Phase I final report follows: 

A theoretical analysis was performed on the relationship between 
the image projected by a prime lens or "fisheye" lens and the original 
object plane. The results verified that the distortion present in a 
fisheye image can be removed for a region of interest by using a direct 
transformation. The algorithm for transformation of the fisheye 
circular image onto a flat object plane provides a forward calculation 
and mapping of the object plane onto the fisheye image without the 
need for iterative calculations. The parameters for the algorithm are 
determined from a knowledge of the digitized image radius alone. 
Rotation of axes allow pan and tilt over a 2% steradian field of view. 
The transformation algorithm is defined in terms of an orthogonal set 
of axes in object plane coordinates mapped into a circle within an 
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orthogonal set attached to the image plane. The final equations as 
developed in the Phase I report are presented below. 

Ru (1) 
-yij2+v 2 +m 2 R 2 



R(vcosp-mRsinp) 
Y = V u 2+v2 + m2R2 



Where: 

R m radius of image circle in pixels 

m - magnification 

u,v - object plane coordinates 

x,y = transformed image plane coordinates 

p m zenith angle 

These two equations represent a mapping rule that maps the object 
plane image point with coordinates (u.v) onto the image plane with 
coordinates (x,y). In the equations, R is the radius of the .mage circle 
in pixels as projected by a fisheye lense onto the CCD array image 
sensor. The zenith angle or tilt angle measured from the normal axis of 
the camera is given as p and varies over an angle of 0° to 90°. The 
equations also provide the capability to zoom by varying m. A value of 
m-1 corresponds to the object plane being placed tangential to the 
effective fisheye image sphere with radius R. Increasing values of m 
provide image magnification or the effect of zooming in on the region 
of interest. The addition of a rotation of axes in the image plane 
provides the capability to pan over 360° angle and is implemented in 
the two following equations. 

x' = x cos 9 - y sin 9 + xoffset (3) 

y' = x sin 0 + y cos 9 + yoffset (4) 



Where: 

x' = horizontal pixel address 

y' - vertical pixel address 

9 m rotation angle within image plane 
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Equations 1, 2, 3 and 4 provide the capability to pan, tilt, and zoom 
anywhere within a 180° hemispherical field of view. 

The hardware design of the transformation algorithm uses video RAM 
memory to perform the transformation in real-time. A set of 
transformation buffers address each pixel of the CCD frame ^ buffer 
during frame transfers. The XY transformation buffers ekm.nate the 
need for calculation of each pixel map in real-time. Once the map has 
been calculated and stored in the transformation buffer, it remains 
constant until a new viewing direction is needed Transform map 
updates can occur simultaneously and asynchronously with the display 
readout and update functions. In actuality, three functions associated 
with the image transformation occur independently - the readout of the 
CCD image sensor into the frame buffer, the display update or mapping 
of the CCD frame buffer to the transformed image, and the .update ot 
the transformation map. A high speed digital signal processor 
performs the calculations and stores the pixel address map in the 
transformation buffers when the view orientation is changed. 

It is apparent from this initial effort, that a digital signal processor 
can accomplish the necessary transformations from the fisheye view to 
the desired pan and tilt view in approximately 1/3 of a second By 
creating a vectored table look up between the fisheye view and the 
desired view the display rate can be accomplished at 30 Hz non- 
interlaced or 60 Hz interlaced rates. After computation of the table 
look up. transformed images at a normal rate of 30 frames per second 
can be achieved to provide normal views of the remote environment. 

Further effort in the specification of a "smart" camera interface 
completed the Phase I activity. The concept of the smart camera is one 
that can be communicated with and operated in a number of states. 
Control of the frame rate, the image resolution, the image type (such as 
frame differencing and vector quantization), and self-d.agnos.ng 
camera functions are all key features of the smart camera concept. An 
additional opportunity that a smart camera interface addresses is the 
limited communications bandwidth that is available in most freespace 
remote applications. 

The smart camera supports operating modes that meet the demands 
of the viewing application. Central views can be provided at high 
resolution and full frame rate, while peripheral views can be 
transmitted less frequently and at a lower resolution. Camera 
operating modes can be shifted at the request of the operator or 
supervisory computer to meet the present needs of the task, me 
resulting system gives maximum viewing capability with minimum 
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communications requirements, a key consideration for future space 
manipulation systems which will rely on freespace transm.ss.on m 
already crowded frequency bands. 

In summary, the Phase I efforts resulted in the following significant 
accomplishments: 

1 . Design and fabrication of a passive positioning method for 
cameras; 

2. Selection of a camera and lens configuration for the system; 

3. Design of a self-contained pan and tilt mechanism for the 
camera; 

4. Fabrication of the mechanical positioning and orientation 
system; 

5. Testing of the system for functionality and ease of use; 

6. Review of available literature on fisheye pattern processing; 

7. Selection of hardware components for an electro-optical 
system; 

8. Detailed conceptual design of an electro-optical pan/tilt; 

9. Specification of a smart camera interface; 

10. Conceptual design of a smart camera interface; 

11. Theoretical development of image transformation algorithm; 

12. Software validation of algorithm; 

13. Development of software application for analyzing fisheye 
images, and; 

14. Preparation of the final report. 

All major objectives of the Phase I activity were met within the 
time schedules. The conceptual design of the electro-opt cal carnera 
provided the theoretical background to continue Phase II efforts and to 
consider the actual design and implementation of the all electronic 
hemispherical viewing system. The conceptual design analysis of the 
electro-optical camera demonstrated the feasibility of the 
implementation. Many of the major elements necessary to complete the 
design are available off the shelf. The key component of importance is 
the new CCD image sensor recently developed by Toshiba Corp. The new 
sensor is expected to be available well within the time frame of the 
Phase II effort. 

The fundamental conclusion of the Phase I work is that an electro- 
optical pan/tilt/zoom camera system is feasible, realizable, and offers 

... 11 

TeleRobotics International, Inc. 



Hectic-Optical Camera System 

a solution needed to advance remote telerobotic implementations. The 
demonstration of the mechanical implementation confirmed our 
assumptions that a smaller package would be useful m the confined 
demote environment. The success of the algorithm the und.stor^t.on 
of a fisheye image gave confidence to our assertions that this could be 
accomplished with reasonable computational systems Our 
development of a conceptual design that located the major electronic 
elements and analyzed the timing requirements leads us to bel.eve that 
a unique and serviceable electro-optical pan/tilt/zoom viewing system 
is realizable by our company in the Phase II effort. 

4. Phase II Technical Objectives and Approach 

The realization and delivery of a "smart" electro-optical 
pan/tilt/zoom remote camera system for passive viewing in. remotely 
operated environments is the major goal of the Phase II research and 
design effort. The electro-optical camera concept provides a compact 
and innovative means of actively viewing the entire manipulator 
workspace. The Phase II effort will extend theo r et '^ 
algorithmic, and conceptual developments provided in the Phase effort 
to a hardware implementation of an electro-opt.cal pan/t.lt/zoom 
camera Further developments for the communications interface 
providing a variable scan rate, variable resolution capability, and 
multiple simultaneous views from the same camera will be 
implemented as the Phase II effort proceeds. The major technical 
objectives are outlined below. 

OBJECTIVE- Stable long-term performance of orientation and camera 
lens controls. APPROACH: Due to the digital implementation no 
calibration is ever needed for the orientation system (i.e., no analog 
sensors, and no gear mechanisms to periodically adjust or lubricate). 
The fully digital camera system also provides long-term stability from 
the imaging standpoint. Electronic gain rather than mechanical iris 
adjustment can control the iris function and the infinite depth of . eld 
of the fisheye lens eliminates the need for an active focus mechanism. 
The varied pixel scan capability automatically provides zoom with no 
moving parts. 

OBJECTIVE: Minimize the size of the unit and make all orientations 
accomplished internally with no moving parts. APPROACH: The use o 
the electro-optical pan/tilt/zoom methodology conceived m Phase i 
will be implemented in a compact package that has no moving parts a 
digital signal processor will be developed to transform the 
hemispherical image into useful views. 
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OBJECTIVE- Provide a 10X zoom capability without a motorized 
zoom on the camera. APPROACH: By using a high resolution sensor and 
the algorithm developed in Phase I, a ten times zoom capa Mrty «n ' be 
supported with single pixel resolution. Higher magn.f. cat.ons (20-40X 
can be accomplished by reducing the output resolution. The concept 
uses selective pixels to obtain a view to the magnification required on 
a tower resolution viewing monitor (i.e. the 1000*1000 CCD array ,s 
scanned and zoomed as needed to provide the 350 line NTSC standard 
resolution output associated with most TV monitors). 

OBJECTIVE' Develop a user interface to the system that is 
responsive and easy to operate. APPROACH: The electro-optical 
camera system must be easy to operate including func ions of pan bit. 
zoom, and operating mode. An operator interface w.ll be deve oped 
during the course of the effort providing direct access to these 
capabilities using a graphical interface. The design goal is to provide a 
camera that can be selected and controlled with minimum operator 
learning and effort. A straightforward man-machine interface 
approach will be utilized that provides high level interface control of 
all camera functions. The operator should be able to select a viewing 
direction by simply touching a region on a control screen Icon based 
control functions and draggable scroll bars can provide zoom and 
rotation within the hemispherical field of view. 

OBJECTIVE' Develop a "smart" camera interface that supports the 
usage of several of these cameras on a single communications channel 
APPROACH- A communications interface for modal operation including 
varied frame rate, varied resolution, frame, difference transmission, 
and camera status transmission will be developed to provide a system 
compatible with NASA's long-term needs. This will reduce 
communications requirements and cable handling considerations. 

OBJECTIVE: Reduce the signal, power, and drive cabling required to 
operate a remote camera. APPROACH: A reduction in the amount of 
signal cabling and power consumption will be gained by the elimination 
of the motion drives and motion sensors. Pan and tilt axis drives and 
motorized zoom, focus, and iris needed to completely control the 
camera view are eliminated by the electro-optical panA.lt/zoom 
system proposed. This also results in cost savings because of the 
reduction in the instrumentation and electronic interfaces. Positioning 
accuracy is not lost because the electro-optical camera view can be 
aimed with a minimum resolution of 0.2° electronically. 

OBJECTIVE- Replace multiple cameras with a single electro-optical 
camera system. APPROACH: A dual ported video memory with the 
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capacity for providing dual split-image transforms simultaneously will 
be implemented to allow tracking of multiple views using a ^ single 
camera. Reducing the number of cameras m a given installation thus 
provides another means by which the electro-optical v.ew.ng system 
will minimize installation costs. 

At the completion of the Phase II effort a prototype electro-optical 
camera system ready for commercialization will be available and the 
first unit will be delivered to the NASA sponsor. Since this will be the 
main objective of the effort, a description of the major technical 
components is provided along with a block diagram to describe the 
approach. The camera will provide electronic v.ew.ng within a 
hemispherical field of view. The viewing angle ( pan and til ) and 
image zoom will be accomplished by electronic means. Added features 
of the camera will allow rotation of the image under software control 
as well as unique commands to provide features such as region zooms 
and split-image viewing. A block diagram of the electro-optical 
camera is shown in Figure 5. The camera consists of the following 
main functional blocks: 

Fisheye equipped CCD camera 
CCD frame buffer block 
Translation buffer block 
Algorithm processor block 
Data compression block 
High speed communication block 
Command channel communication block 
Peripheral interface block 
The CCD interface consists of the CCD image sensor interface and a 
CCD image frame buffer for intermediate storage of the complete CCD 
image The image conversion block is responsible for und.stort.ng the 
fisheye circular image and providing serial pixel readout to the data 
compression block. The data compression block transforms the 8-b.t 
serial pixel data from the CCD frame buffer to a compressed form and 
temporarily stores the data in a hold buffer for later communication. 
The image and control processor consists of a high speed digital signal 
processor that calculates the X and Y pixel conversion addresses that 
are stored in the image conversion block. A two link serial 
communication interface provides a networked camera system using a 
high speed video transmission link and a low speed control link. 
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Figure 5 - Electro-Optical Camera Block Diagram 
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There are two important keys to the implementation that wil 
guarantee its success. First, the Digital Signal Processing (DSP) 
implementation of the developed algorithm will significantly speed the 
deration assuring the conversion time of less than 1 second (we 
actually anticipate that this conversion can be done in less than 0.3s 
but only hardware verification will prove this). Secondly, using a table 
look up transformation system, the 30 Hz throughput of the viewing 
system transformation can be guaranteed for a continuous image. The 
table look up is a novel approach that will assure that the camera 
response is satisfactory for human operations as well as computer 
vision autonomy. 

An operator control station will be implemented that provides 
control of the camera network and selection of specific v.ews The 
operator control station will consist of a personal computer with full 
color display capabilities to provide complete icon based graphics for 
camera control. The Phase II goal is to provide a single man-machine 
interface platform that enables control and display of the camera 
views on the personal computer monitor. The frame rates of each of 
the cameras will be controlled to obtain the best viewing situation and 
maximum available bandwidth utilization for the data communication 
channel Each camera can be asked to provide views at varying frame 
rates The operator display station would provide a constant display in 
between frames for cameras that are transmitting at slower frame 
rates. 

5. Phase II Statement of Work 

The electro-optical camera design provides the opportunity to 
advance the state-of-the-art in high speed digital image processing 
and video data bandwidth compression. The major objectives of 
Section 4 are all related to successful implementation of the electro- 
optical pan/tilt/zoom camera system shown in the block diagram of 
Figure 5 The worktask milestones revolve around implementation ot 
the detailed designs that support the block diagram of Figure 5. These 
tasks are outlined below. A performance schedule and description ot 
the task goals have also been included. 

(1.0) Procure and modify high resolution CCD camera with fisheye 
mounting; 

(2.0) Develop algorithm software control features; 
(3.0) Design and prototype command channel link; 
(4.0) Design and prototype CCD frame buffer block; 
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(5.0) Design and prototype translation buffer block; 

(6.0) Design and prototype algorithm processor block; 

(7.0) Development of man-machine interface platform; 

(8.0) Design and prototype data compression block; 

(9.0) Design and prototype high speed communication link; 

(10.0) Design and prototype A/D interface to CCD camera. 

Figure 6 gives the schedule and milestones for the Phase 

described herein. 



II effort 



Task 

1.0 
2.0 

3.0 
4.0 
5.0 
6.0 
7.0 
8.0 
9.0 
10.0 



on — o 



12 



6u — o on — O DO 

(month) 24 



^ Milestones 
O Reports 
□ Technical Exchange 



Figure 6 - Schedule for development of the electro-optical pan/tilt/zoom camera 

system. 

The milestones for each of the subtasks are: 

1 ) High resolution CCD and fisheye operational 

2) Final software including data compression complete 

3) Command link functional 
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4) Frame buffer block functional 

5) Translation buffer block functional 

6) Processor with algorithm functional 

7) Man-machine interface operational 

8) Compression block operational 

9) Communications link and entire system operational 
10) A/D interface to CCD operational. 

In the following sections, each of the main tasks outlined above are 
described as specific subtasks. A brief description of the performance 
objectives for each major technical task is included. 
(1.0) Procure and Modify Camera with Fisheye Mounting 

The CCD camera is one of the main components to procure. The most 
appropriate choice would be to obtain the camera from Toshiba t who 
plans to incorporate the high definition CCD array sensor into a t camera 
for the High Definition Television (HDTV) market .n Japan. S nce HDTV 
is successful and a large effort is being placed °n the p " D JV 
development, this camera should be available well w.thm the Phase II 
time frame. 

The fisheye equipped camera is one of the critical components of the 
system ttS would need to be constructed. There are several 
ma^L^rs that make fisheye lenses and fisheye lens attachments 
for 35 mm format cameras. Although fisheye lens adaptors are 
available for around $100, fisheye lens adaptors are not true fisheye 
lenses and would not provide the necessary features and image quality 
One of the first activities in the development would be the purchase of 
the lens and a camera interface so that further algorithm testing could 
be performed early in the program. 

A lens reduction element is necessary in order to adapt the fisheye 
lens image onto the CCD array sensor. The fish eye^ns produces a 23 
mm effective diameter circular image on normal 35 mm film The OOU 
Censor has a useful area of about 10.5 mm so that a /2 reduction 
would need to be obtained. This reduction can be accomphshed through 
a 3 lens arrangement consisting of two reducing lenses and a field lens^ 
This would have to be incorporated between the f.sheye lens and the 
CCD camera. A conceptual picture of the fisheye lens equipped camera 
is shown in Figure 7. 
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Flsheye Lens 



CCD Camera 




Image Reduction 



Figure 7 Fisheye Equipped Electro-Optidal Camera 

Prior to the availability of the camera, efforts TOn ^" tra j e "Jl 

the electronics design of the frame buffer ^ d t ^ n ^ on d . b ^rsigna 
interface to the camera and the commun.cation and digital signal 
orocessor designs. The interface to the CCD camera consists o a flash 
converter interface and timing logic. The A/D converter mterfaDa 
would be defined separately from the camera so that design progress 
cou I be n^ade on ?he memory buffers and digital signal processor 
Sace The major activities associated with the camera fabrication 
task are: 

1.1 Contact manufacturer for detail on CCD sensor 

1 .2 Procure camera with high definition CCD sensor 

1.3 Procure fisheye lens 

1.4 Design lens reduction interface 

1.5 Procure lens reduction components 

1.6 Fabricate camera 

The final result of this activity is a fisheye f^P^f^J^ 
provides a digitized 180° fisheye view. The interface to the CCD array 
will be provided by either modifying the camera electronics so that the 
pixel JZ™ be captured directly or by adding an external interface 
to digitize the analog information from the camera. 
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(2.0) Develop Algorithm Software Control Features 

The electro-optical camera provides the opportunity to investigate 
new methods for remote viewing. The viewing conditions are 
completely determined by the transformation map stored in the 
Sormation buffers. Since these can be controlled completely by 
Xare an endless number of possibilities are at hand. Before a fully 
functional camera is available, all of the possible display conditions 
can be investigated by using a fisheye equipped 35 mm camera and a 
high resolution digitizer. The digitizer would be used to _d.g. tize -the 35 
mm format pictures into the expected resolution of the CCD array 
sensor for analysis. The selected digitizer would offer a 256 level or 8 
bit digital gray scale picture resolution to match the requirements of 
the electro optical camera. The digitizer would be interfaced to a 
personal computer so that a full digitized bit-mapped image can be 
stored for analysis. The Phase I software would be extended to provide 
the camera functions and added features. The benefit of this approach 
is the ability to write the software and define the feature control 
before the camera is fabricated. These software efforts would be 
carried out in parallel to the electronics design. The major act.v.t.es 
associated with this task are outlined below. 

2.1 Procure testbed fisheye lens 

.2.2 Procure 35 mm camera and mounting 

2.3 Procure digitizer 

2.4 Develop software interface to digitizer format 

2.5 Extend Phase I software to digitizer format 

2.6 Develop software feature controls 

2.7 Develop man-i.^ciiine interface concept 

The main initiatives for this task are to examine different ways for 
controlling the viewing format and to investigate operator man- 
machine interface functions for the final system integration. 
(3.0) Design and Prototype Command Channel Link 

The command channel provides the low speed communication link to 
each of the cameras on the network. Its purpose is to provide contro 
information to the camera. The control information would consist : of 
viewing direction ( pan and tilt ), zoom or magnification, .mage 
rotation angle, frame rates and frame attributes, and control 
information for auxiliary functions. Peripheral control of auxiliary 
functions are included for convenience in viewing and consists of 
on/off control of background lighting and lighting levels. The command 
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channel can be a relatively slow communication link. Possible choices 
are RS-422 differential with camera to camera repeaters, coaxial and 
fiber optic The coaxial cable and fiber optic communication interfaces 
are attractive because cameras can be passively connected to the 
network This allows cameras to be added or removed from the 
network during operations without disturbing the working network. 
The overall objective of this task can be broken down into the 
following activities: 

3.1 Selection of appropriate physical link 

3.2 Design physical interface 

3.3 Procure electronics and hardware connections 

3.4 Assemble and test physical interface 

At the completion of this task, a physical interface will be available 
for testing that provides passive connection onto a multiple camera 
network. 

(4.0) Design and Prototype CCD Frame Buffer Block 

The CCD frame buffer consists of a 1024 by 1024 pixel memory 
buffer for storage of the CCD image as it is read out from the CCD 
imaqe sensor. The frame buffer is implemented as 8 bit planes to give 
a minimum gray scale dynamic range of 52 dB. This would be necessary 
to eliminate the possibility of false contouring in the image. Since the 
reported dynamic range of the CCD sensor developed by Toshiba is over 
70 dB a very good gray scale picture should be attainable with minimal 
contou'ring and noise effects. The image buffer would be updated at a 
30 Hz or 30 frame per second rate. This would represent the minimum 
necessary frame rate to provide a flicker free image in the final 
display Readout of the CCD frame buffer can be accomplished as either 
a 30 Hz non-interlaced or 60 Hz interlaced scan implementation 
depending on the display monitor. The major activities associated with 
this task are: 

4.1 Selection of video RAM devices 

4.2 Design frame buffer and interface 

4.3 Procure electronics parts 

4.4 Prototype and test design 

At the completion of this task, a CCD frame buffer and interface 
will be available for testing with the remainder of the camera 
electronics. 
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(5.0) Design and Prototype Translation Buffer Block 

The translation block provides the real-time undistorted image 
conversion from the contents of the CCD frame buffer block. The 
translation buffer block consists of a horizontal and vertical 
translation buffer and a video system controller interface. The 
horizontal and vertical translation buffers provide the real-time 
window into the CCD buffer for undistorting the circular fisheye image. 
The translation buffer readout and addressing rates are controlled by a 
video system controller (VSC). The VSC performs the display update and 
refresh cycles for the translation buffers automatically. The VSC is 
controlled and programmed by the digital signal processor and is 
reprogrammed only when the frame scan rates are changed. The 
translation buffer design consists of the following activities: 

5.1 Selection of video RAM devices 

5.2 Selection of video controller 

5.3 Design translation buffer and interface 

5.4 Design VSC interface to translation buffer- 

5.5 Procure electronics parts 

5.6 Prototype and test design 

At the completion of this task, a translation buffer and interface 
will be available for testing with the remainder of the camera 
electronics hardware. The algorithm processor interfaces directly to 
the translation buffer block through the video system controller. The 
VSC provides the programmed readout of the the translation buffer and 
subsequent address look up of the CCD frame buffer. 

(6.0) Design and Prototype Algorithm Processor Block 

The algorithm processor block is responsible for providing the 
updated transformation map for the XY translation buffers. In this 
task, several different digital signal processing options must be 
considered. Several vendors offer special purpose high speed VLSI 
digital signal processing circuits. The major considerations are the 
cost, power consumption, speed, and packaging size of the final camera. 
It is desirable to use a single-chip solution for the algorithm 
processor. There are several commercially available digital signal 
processors that could perform the algorithm calculations at greater 
than a 1 Hz rate. Most notably are the Analog Devices ADSP-2100, the 
Motorola DSP56001, the Texas Instruments TMS320 series, and the 
Harris RTX2000. The selection of the processor core must include 
consideration for programming tools and language support and the 
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feasibility of communication with the command channel for camera 
control. The design of the algorithm processor consists of the 
following basic design activities: 

6.1 Selection of processor 

6.2 Procure development system 

6.2 Design processor core 

6.3 Procure electronic components 

6.4 Prototype and test design 

6.5 Develop test software for hardware evaluation 

The results of this task are to provide a processing core that can 
provide the algorithm transformations in real-time for camera motion 
control The processor will communicate with the control channel to 
receive commands for electronically controlling the camera view and 
frame rates. Also, the processor will provide the control inputs to the 
data compression block to control the bandwidth compression 
algorithm and frame rates. The processor core can be tested as an 
entity before it is interfaced to the CCD frame buffer and 
transformation buffer blocks. 

(7.0) Development of Man-Machine Interface 

The man-machine interface is provided so that the operator can 
control the camera network using a high-level graphics and objec 
based interface. The man-machine interface would consist of a general 
purpose personal computer with a high resolution monitor and 
appropriate graphics and scanner interfaces. The interface will provide 
a graphical means for the operator to select and magnify views and o 
change the operating mode of the viewing system. This task would 
involve the following objectives: 

7.1 Procure computer system 

7.2 Procure monitor 

7.3 Procure software development language 

7.4 Procure graphics tools 

7.5 Develop menu control approach 

7.6 Develop software menu application 

7.7 Develop status presentation displays 

7.8 Refine software approach using final camera system 
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The end result of this task is to provide a simple man-machine 
interface with graphics based menus for operator control of cameras in 
the vision control network. 

(8.0) Design and Prototype Data Compression Block 

The data compression block provides the video bandwidth 
compression of the undistorted image before transmission over the 
network Since there is a limited amount of data bandwidth in a 
freespace environment situation or even coaxial or fiber optic cable 
communication links, the desire is to achieve a high degree of 
bandwidth compression using several different approaches. Research is 
necessary to determine the best possible methods for optimum 
bandwidth compression of real-time video information. There are 
several viable alternatives to video data compression with the amount 
of bandwidth reduction heavily weighted by the type of picture 
information being processed. The number of digital bits per sample 
required is determined by the problems of false contouring rather than 
by proper gray scale rendition. False contouring leads to edges and 
gray-scale transitions that were not present in the original picture 
Two methods that look most promising at this time are the Differential 
Pulse Code Modulation DPCM and Vector Quantization approaches. These 
will have to be investigate in more detail before a particular choice is 
made. The most important activities associated with this effort are 
outlined below. 

8.1 Selection of bandwidth compression approach 

8.2 Design bandwidth compression high level approach 

8.3 Design bandwidth compression low level and interface 

8.3 Procure electronic hardware 

8.4 Prototype and test design 

8.5 Incorporate design into final system 

The data compression block provides a reduction in the bandwidth or 
amount of data to be transmitted over the communication link. Once 
the data is reduced, the high speed communication block can transmit 
the data at selected frame rates up to the maximum necessary rate of 
30 frames per second. The goal is to be able to multiplex via time 
division and/or inter-frame multiplexing several camera views using a 
single network link. 
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(9.0) Design and Prototype High Speed Communication Link 

The purpose of the high speed communication link is to provide time 
multiplexed frame information from several cameras using a single 
high speed communication link. When multiple cameras are to be 
utilized in a single remote viewing system, methods of sharing the 
same communications channel can be applied to minimize the 
communications overhead for freespace. These include selective frame 
rate transmission, image bandwidth compression, and variable 
resolution imaging. In selective frame rate transmission, the frame 
rate from a particular camera is controlled and can be set to lower 
rates when the view is not needed for real-time control. For instance, 
the rate on one camera could be set to 1 frame per second if it is a 
background picture needed for overall position reference, or if its 
information is only of small importance. Camera views not needed for 
the particular remote task could halt transmissions until they are 
needed. In this manner, the most important camera views needed for 
the particular task could be time multiplexed over a single high speed 
network. The major activities associated with this task are: 

9.1 Select communication physical channel 

9.2 Design multiplexing scheme 

9.3 Design transmit and receive interfaces 

9.4 Procure electronic components 

9.5 Prototype and test design 

9.6 Integrate into camera system 

The result of this task is to provide a communication link that will 
enable several cameras to be networked using a single physical 
communication network. 

(10.0) Design and Prototype A/D Interface to Camera 

The CCD camera provides a HDTV composite signal for the high 
definition television standard. This signal must be sampled and 
digitized by a high speed flash A/D converter at the HDTV scan rates. 
Alternatively, the A/D interface could be integrated into the camera 
electronics to digitized the pixel information from the CCD sensor 
interface electronics. The CCD sensor provides two parallel pixel 
output paths for each horizontal scan line. This reduces the necessary 
conversion rate of the A/D converter to a maximum of 37 MHz which is 
within the sampling rate of commercially available flash A/D 
converters. This task consists of the following major activities: 
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10.1 Determination of CCD interface method 

10.2 Selection of A/D 

10.3 Selection of A/D interface buffer amps 

10.4 Procure electronics hardware 

10.5 Design, prototype, and test A/D interface 

10.6 Integrate into final system 

The A/D interface will be implemented as a dual pixel interface. 
The CCD sensor provides a dual channel readout of the horizontal CCD 
shift register so that slower clock signals could be used while 
maintaining a 30 Hz frame update rate. The dual channel A/D interface 
will provide a dual channel write capability to the CCD frame buffer so 
that the CCD frame buffer will have to be split into an odd and even 
pixel arrangement. This odd and even pixel arrangement will have to be 
considered in the design of the CCD frame buffer block and the XY 
translation buffer block designs. The address look up and pixel readout 
of the CCD frame buffer into the data compression block is still on a 
pixel serial basis so that its design is not affected. The address look 
up is simply implemented by employing the least significant bit as an 
odd/even switch between halves of the CCD frame buffer. This 
represents a very simple solution to the CCD pixel interleaving from 
the sensor. 

6. Key Personnel 

Dr. H. Lee Martin - Principal Investigator 

SUMMARY OF CAPABILITIES 

Dr Martin has over 8 years of experience in the design, development, 
testing, and implementation of human controlled manipulation systems 
for hostile environments. He has exceptional hands on experience with 
this type of equipment having learned from systems integration on four 
completely different teleoperated manipulators while employed at the 
Oak Ridge National Laboratory. He brings TRI a knowledge of 
mechanical engineering from his educational background that has been 
augmented with extensive human factors and electrical engineering 
experience in digital systems. His ability to find simple, alternative 
solutions to difficult problems has been his forte. 

Seeing the potential for growth in the area of robotics applications 
in remote and/or hostile environments, Dr. Martin founded TeleRobotics 
International, Inc. in 1986 to provide control and mechanism design 
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products and services to the industry that is evolving around this 
technology. 

FDUCATION: 

B.S.M.E., Univ. of Tennessee, 
M.S.M.E., Purdue University, ' . 
PhD. in M. E., Univ. of Tennessee, 

ENGINEERING EXPERIENCE: 

President and Founder of TeleRobotics International, Inc.: 

Started in J 1 to provide design and consultant services in 
teleoperatedSnd robotic manipulators and digital control systems. 

Development Engineer at Oak Ridge National Laboratory: 
Initiated and led the TeleRobotics Task Force (1985), a group formed to 
find new research funding opportunities for teleoperation 
developments. Lead control engineer on the Advanced Servomanipulator 
development project T- .') a force-reflecting manipulation 
system built for hostile environments. Prior to this activity 

he developed software and algorithms,? for. the world's first 
digitally controlled servomanipulator (the Central Research Lab. Model 
M-2 ) Supervised the research of 3 graduate students and over a dozen 
cooperative education students in addition to several development 
engineers. 

Consultant for REMOTEC Corp.: Developed controls for a remote 
maintenance device and providing controls and system integration 
expertise for this firm£~ 3 

Research Assistant at Purdue Univ.: £7 J Developed 
battery state-of-charge indication methods for electric vehicles. This 
effort later led to a patented device (US patent 4390841) for battery 
monitoring. 

Engineer's aide for T.V.A.: (T JJesigned nuclear power 
plants. 

PROFESSIONAL INVOLVEMENT: 

* National Young Professional Engineer of the Year, U.S., 1988 
(Society of Professional Engineers). 

* Founding Chairman of the Knoxville/Oak Ridge Subchapter of 
Robotics International 

* Recipient of ImVftri"' ^search magazine's IR100 award in 
f J for development of the first digitally controlled servo- 
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manipulator (noted as one of the 100 outstanding new technical 
product's ^ 

* Editor of a technical paper collection TPleopeiafed Robotics for 
Hnstilfl Environments published nationally by the Society of 
Manufacturing Engineers in 

* Organizer and lecturer of a tutorial on Remote Handling. This 
event held in conjunction with the American Nuclear Society 
Topical Meeting attracted 95 participants from 8 countries. 

* Technical review committee and session go-chairman for ANS 
Topical Meeting on Remote Handling in £_ . J 

* Outstanding Young Manufacturing Engineer, Q ^ Society of 
Manufacturing Engineers (12 nationally). 

* Young Professional Engineer of the Year, State of Tennessee, 

Registered Professional Engineer in Tennessee. 

* Member of the American Society of Mechanical Engineers. 

* Member of the Society of Manufacturing Engineers, Robotics 
International. 

Steven D. Zimmermann-Research Associate 

SUMMARY OF CAPABILITIES: 

Mr Zimmermann has a strong background in the design and 
fabrication of analog and digital electronics He has independently 
developed and designed several control components for remote 
manipulator systems including a self-contained control pendant for 
operator interface with remote auxiliary equipment, a joint processor 
network for robotic sensing and control, and a communications network 
for telerobotic operations. His expertise in surface mount technology 
assures that his solutions are not only functional, but packaged as 
small as possible with today's technology. His experience with real- 
time controls and his ability to integrate software and hardware allow 
Mr Zimmermann to efficiently accomplish seemingly impossible tasks_ 
His balance between algorithms, software, and hardware is unique and 
exceptional. Mr. Zimmermann is considering the opportunity ot 
developing a Master's thesis around this Phase II proposal utilizing 
these algorithmic and hardware developments as the subject ot a 
thesis on real-time image enhancement. 
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FDUCATION: 



B.S.E.E., University of Tennessee,^ ^} 
M.S.E.E., Candidate, University of Tennessee 

FNftlNEERING FXPERIENCE: 

Electrical Engineer, TeleRobotics International, Inc. 

Responsible for the development and im P , « mentation Fff 0 J rts r °ha V e 
control systems, software support, and circuit design. Efforts have 
Sed lhl development of an Application Specific . Integra ^ed C^rcu t 
for manipulator joint control, development of transf0 ^ mat '°" 
algorisms for remote viewing, and development of commumcat.ons 
links for manipulator operations. 

Development Engineer, Oak Ridge National Laboratory, 
Deve,0 S m Hardware 9 /software design of ro botic and raaster/s ave 
Wvomaniplflator systems including the Advanced Servomampulator 
and the Laboratory Telerobotic Manipulator. 

_ Summer Engineering Intern, Texas lii.trumefit* C^mmor 
lT_7Hardware design of very high speed integrated c.rcu.ts (VHSIC) 

for"~military processing applications. 
^-Student Engineering Assistant, Technology for Energy Corp. 

Worked in^roduct development performing hardware design and 
development for nuclear power plant instrumentation. 

PROFESSIONAL INVOLVEMENT: 

Member of Tau Beta Pi 

Member of Eta Kappa Nu 
Paul E. Satterlee, Jr. - Research Associate 

SUMMARY OF CAPABILITIES 

Mr Satterlee is an expert in the development of real-ti me control 
systems for unique applications. He has the ability ^ ^tch ateo^m 
requirements, input/output needs, and computational hardwa. re 
capabilities to build systems that function extremely we I. This 
aWity has been gained from the application of digital control systems 
to several robotic and teleoperated manipulator systems including 
hardware architecture, design of components, and construct^ i of 
svstem software. He has become an expert at the programm ng 
Sage FORTH and has developed several libraries of functions related 
to robotic and teleoperated manipulator controls. 
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Mr Satterlee was the sole control system hardware designer off the 
award winning Model M-2 manipulator system at the Oak Ridge National 
Laboratory. His present research interests are in the application of 
artificial intelligence to real-time control applications. 

B.S.E.E., Louisiana State University,! 
M.S.E.E., Louisiana State University,^ 

FNG1NEERING FXPERIENCE: 

Vice President, Computer Systems^ TeleRobotics 
International, Inc. Founding partner in TRI ^ J to explore 
research in the area of future manipulator systems. 

Telerobotic Systems Consultant, £ ^ developed 

control system hardware and software for a mobile surveillance robot 
and master/slave servomanipulators for Remote Technology Corp. in 
Oak Ridge, Tennessee. ^ ^ 

Staff Engineer, Oak Ridge National Laboratory, f - J 
hardware/software design of robotic and itaSter/slaye 
servomanipulator systems, microprocessor-based smart instruments, 
data acquisition systems, and other process control systems. 

Consultant, University of Tennessee C- J J° ^° a n k 
Ridge National Laboratory Instrumentation^and Controls Division, 
researching Computer Aided PLC Programming. 

Graduate Research Associate, University of Tennessee, 
C J designed and built prototype microprocessor-based, 

^programmable data collection platform under NASA contract. 

Engineer, WWL TV-FM-AM broadcast stations, New 
Orleans, Louisiana, worked as cameraman, video controller, video tape, 
projection, and transmitter duty. 

PROFESSIONAL INVOLVEMENT: 

* Member of Eta Kappa Nu (Electrical Engineering Honor Society). 

* Certificate of Merit for "Best Paper" at the 29th RSTD 
Conference. 

* f 3 Industrial Research IR-100 Award for the development of 

the "M-2 Digital Control for Servomanipulator". 
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pFI FVANT PUBLICATIONS OF INVOLVED TR1 PERSONNEL 

I . "Applying the Novix Computer to Real-time Control of Redundant 
Teleoperator and Robotic Manipulators", Final report, U. S. doe 
contract DE-AC05-86ER80406, £ J 

2 "Design Enhancements for the Advanced Servomanipulator" 

' Final report, U. S. DOE contract DE-AC05-86ER80399, J 

3 "Development of Dual-Arm Robotic Manipulator Control Based on 
Teleoperated Manipulation Methods", Final Report, NASA 
contract NASI -18423. 

4 "Integrated Digital Control and Man-Machine Interface for 
' Complex Remote-Handling Systems," International Topical 

Mo« finn nn Rem o te Systems and Robotics in Hostile 
pnvironments. ^ . J \ 
5. SURBOT: A Surveillance Robot System for Use in Nuclear Power 
Plants, Robot; £ J) Conference, Chicago, IL, 

6 Mobile Surveillance Robot System, Proceedings of the American 
Nuclear Society Topical Meeting on Robotics and Remote 
Handling in Hostile Environments, Salt Lake City, UT, (_ 

7 "Surveillance Robot for Nuclear Power Plants," Proceedings of 
the American Nuclear Socjaty Remote Systems Technology 
Division, San Francisco, Ca.,|. .. 

8 "The State-of-the-Art Model M-2 Maintenance System," 
Proceedings of the American Nuclear Society Topical Meeting on 
Robotics and Remote Handling in Hostile Environments, 
Gatlinburg, Tn., f " *"7 

9 "Control Software Architecture and Operating Modes of the 
Model M-2 Maintenance System," Proceedings of the American 
Nuclear Society Topical Meeting on Robotics ajid Remote 
Handling in Hostile Environments, Gatlinburg, Tn., 

10 "Distributed Digital Processing for Servomanipulator Control," 
Proceedings of the American Nuclear Society Remote Systems 
Technology Division Conference, Washington, D.C., r" ^1 

r j 

II. Electromechanical Three- Axis Development for Remote 

Handling in the Hot Experimental Facility," Proceedings of the 
American Nuclear Society 29th Remote Systems Technology 
Division Conference, San Francisco, Ca.,£ J 
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12. "Robotics-Related Technology in the Nuclear Industry", Society 
of Photo-Opt'^l instru mentation Engineers,/ — -* 

13 "Control and Electronic Subsystems for the Advanced 
' Servomanipulator", / "Wional Tonirgl Meefnn on Robofcs 

^ Rpmnte HenHiTno in Hostile Environments ,^ 

14 "Automatic Camera Tracking for Remote Manipulators" J 
' ^tinnal Topic a l MePtinn on Robotics and Remote Handfinq .n . 

Ho«Tt !| e Environments. . 

15 "An Advanced Remotely Maintainable Force-Reflecting 
* Servomanipulator Concept", ' " Jajional Topical Meeting Qn 

Rnhntir^ and Rftmntft Han d li ng in Ho stile Fnvtronments , ^ 

16 "Joining Teleoperation with Robotics for Remote Manipulation in 
' Hostile Environments", Robots 8 . Rooties International of the 

Society of Manufacturing Engineers,/^ / 

17. "Evaluation of Robotic Inspection Systems at Nuclear Power 

Plants", NUREG / CR-3717, /^ . 
18 "Robotic Design Analysis Based on Teleoperated Manipulator 
' Data Collection", Robots 9 . Robotics International of the Society 

of Manufacturing Engineers./^ - — ' 
19. Tfilpnpprated R nhntirs for Hostile Environments, Editor, Society 

of Manufacturing Engineers,^ J 
20 -Teleoperated Robots - The Manipulation Side of Nuclear Remote 

Maintenance Systems", Sidebar article for IF F F Spectrum ,^ j 

21.^Duty Cycle Analysis of a Human^Controlled Robot", Journal of 
Rohntic Systems p 

22 -Conversion of a Servomanipulator from Analog to Digital 
' CnnW f J7 FFF Internat io n a l Confe rence on Robotics and 

Automation. San Francisco, £1 ^-7 

23 "Recommendations for the Next Generation Space Telerobot", 
Technical Memorandum 9951, Oak Ridge National Laboratory, 
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7. Facilities/Equipment 

TeleRobotics International, Inc. operates a laboratory/office facility 
in Knox County, Tennessee. These facilities will supply the workspace 
and most of the equipment required for this effort. Special support for 
printed circuit board fabrication will be contracted as required for 
prototype development. Design computational facilities are already 
available as are the necessary software design and simulation tools 
needed for this activity (extensive use of automated circuit layout and 
simulation tools available at TeleRobotics International, Inc. will be 
required during this effort). 

The unique requirement for the programming of a Digital Signal 
Processor will require the purchase of a DSP development system to 
support this specific effort. One station is estimated in the budget. 
Due to the extremely fast response of the video circuitry required for 
this project, a high frequency response scope is also required for the 
testing portion of this effort. The scopes presently used at our 
facilities do not have adequate response ranges for this purpose, 
therefore a high frequency bandwidth oscilloscope is also estimated in 
support of this effort. Other parts and equipment in the budget are 
directly related to the fabrication of the prototype systems for 
delivery to NASA. 

The majority of the effort is reduction of concept to practice. Since 
we validated the algorithm in the Phase I effort, the only question 
remaining is one of speed of the electronics. Our efforts will therefore 
focus on design, implementation, and testing. Special equipment 
requirements are limited to those described above. 

8. Consultants 

Dr Don Bouldin and Danny Newport will consult on this effort to 
provided Application Specific Integrated Circuit tools to enable the 
designs developed in the Phase II effort to be miniaturized for 
commercialization. They are experts in the field of Computer Aided 
Design Tools for the development of ASICs and. have supported our 
efforts in the past on other comparable activities. A statement of 
work and an estimate from Dr. Bouldin for development of ASIC tools 
and implementation of an ASIC chip has been obtained for the purpose 
of the budget estimates enclosed. 

9. Budget 

Form 3090-0116 is attached along with supporting break out 
information per instructions of Table 15-2. Additional materials 
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support is also provided for the cost of key components in the design. 
Where possible, cost estimates are based on data sheets and/or vendor 
contact. In specific instances where subcontracted fabrication is 
required, engineering judgement was used to estimate the cost of the 
prototype fabrication based on past experiences. 
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